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INTRODUCTION 


Background 

During  1976,  the  Federal  Aviation  Administration  (FAA)  required  all  commercial 
airliners  to  be  equipped  with  Ground  Proximity  Warning  Systems  (GPWS).  Such  a 
system  is  designed  to  enhance  flight  safety,  particularly  during  the  takeoff  and 
approach/landing  phases  of  flight.  Since  that  time,  Controlled  Flight  into  Terrain  (CFTT) 
mishaps  for  commercial  airliners  have  been  near  zero.  The  Air  Force  similarly  became 
interested  in  GPWS  systems  for  its  potential  life  and  cost  saving  benefits.  Earlier  focus 
was  on  the  development  of  GPWS  and  Ground  Collision  Avoidance  Systems  (GCAS)  for 
fighter-type  aircraft.  GPWS/GCAS  systems  have  been  incorporated  or  are  being 
incorporated  into  the  A-7,  A-10,  F-16,  and  F-ll  1  aircraft.  However,  little  attention  has 
been  directed  toward  GCAS  development  for  cargo/transport/tanker  aircraft. 

From  1970-1989,  United  States  Air  Force  cargo/transport/tanker  aircraft  have  been 
involved  in  thirty-one  Controlled  Right  Into  Terrain  (CFTT)  mishaps.  A  review  of  these 
mishaps  (Rueb  &  Kinzig,  1989;  Appendix  A)  revealed  that  over  50%  of  the  accidents  may 
have  been  prevented  by  an  operable  Ground  Collision  Avoidance  System  (GCAS).  In  an 
effort  to  prevent  similar  mishaps,  the  Air  Force  is  investigating  the  plausibility  of 
incorporating  a  GCAS  system  into  cargo/transport/tanker  type  aircraft. 

Air  Force's  Strategic  Air  Command  (SAC  Statement  of  Need  (SON),  1987)  and 
Military  Airlift  Command  (MAC  SON,  1987)  have  established  a  user  need  for  a  large  body 
aircraft  GCAS  system.  Accordingly,  Air  Force  Logistics  Command  (AFLC)  contracted  the 
Cubic  Corporation  of  San  Diego  to  develop  a  generic  GCAS  system  for  implementation 
into  the  cargo/transport/tanker  fleet.  This  report  is  Part  1  of  a  two-part  study  conducted  to 
evaluate  the  effectiveness  and  accuracy  of  the  Cubic  GCAS  algorithm. 

GCAS  System  Design 

GCAS  systems  generally  use  software  algorithms  to  compute  the  altitude  for 
emergency  warning  onset  based  on  aircraft  sensor  inputs  such  as  airspeed  and  altitude. 

Two  considerations  are  important  in  designing  the  system.  First,  emphasis  is  placed  on 
minimizing  false  alarms/nuisance  warnings.  These  occur  when  the  pilot  has  not  committed 
a  ground  clearance  error,  yet  a  warning  is  generated.  The  second  concern  requires  a 
warning  to  be  generated  whenever  a  ground  clearance  error  has  been  committed. 

Nuisance  warnings  may  be  the  result  of  three  factors.  Erroneous  sensor  input 
could  cause  a  false  alarm  and  is  a  consequence  of  the  algorithm's  level  of  sophistication. 
Another  factor  is  the  algorithm’s  ability  to  predict  pilot  reaction  time.  Pilot  reaction  time  is 
the  elapsed  time  interval  between  the  system's  recognition  of  a  ground  clearance  error 
(warning  initiation)  and  the  pilot's  initiation  of  the  recovery  maneuver.  For  example, 
reaction  time  differences  (predicted  v.  actual)  of  just  one  second  for  a  steep  dive 
(12,000  feet  per  minute)  could  add  an  extra  200  feet  to  the  predicted  total  altitude  loss. 
Finally,  the  algorithm's  inability  to  accurately  predict,  calculate,  or  extrapolate  could  also 
cause  a  nuisance  warning. 

GCAS  algorithms  usually  are  designed  using  one  of  two  approaches.  The  first 
approach  used  by  Cubic's  Ground  Collision  Avoidance  System  (Cubic,  1985)  and 
Fairchild’s  Low  Altitude  Warning  System  (Shah,  1988)  is  based  on  a  set  of  generic 
aerodynamic  equations.  These  algorithms  may  differ  in  the  various  assumptions 
(e.g.,  G-load,  roll  and  pitch  limits,  pilot  reaction  time)  used  to  calculate  predicted  altitude 
loss.  General  Dynamic's  (GD)  Enhanced  Ground  Clobber  uses  the  second  approach.  This 
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approach  uses  algorithms  based  on  form  functions  (sometim- «  ;  •  cc*  ci.  1  ’ '  a.ig 

empirically  derived  by  off-line  simulation.  The  Cubic  GG-.o  i.'gontiuii  evaluated  r.  'Gs 
report  followed  the  first  approach. 

GCAS  Evaluation  Procedure 

The  Cubic  GCAS  algorithm  was  evaluated  in  four  phases.  Throughout  the  four 
phases,  concerns  and  recommendations  were  forwarded  to  the  System  Program  Office  and 
to  the  Cubic  Corporation.  This  resulted  in  modification  of  the  algorithm  prior  to  the  next 
phase  of  the  evaluation. 

Phase  I  efforts  focused  on  the  verification  and  validation  of  the  algorithm.  This 
phase  simply  established  how  well  the  algorithm  predicted  altitude  loss  based  on  current 
inputs.  Phase  II  was  the  Robot  Pilot  Model  phase.  During  this  phase,  a  computer  pilot 
model  tested  the  GCAS  under  different  aircraft  configurations  and  environmental 
conditions. 

The  man-in-the-loop  phase,  Phase  III,  used  a  subset  of  the  configurations  in 
Phase  II  to  determine  the  algorithm's  ability  to  accurately  predict  under  realistic  conditions 
(human  reaction  times).  In  the  final  phase,  current  operational  pilots  flew  mission 
scenarios  based  on  selected  CFIT  mishaps.  This  phase  permitted  an  evaluation  of  the 
Cubic  GCAS  algorithm  under  real  world  conditions. 

Cubic  GCAS 

The  Cubic  algorithm  is  comprised  of  four  main  subroutines  labeled  GCASALRT, 
GCASDIVE,  GCASROLL,  and  GCASLAND.  GCASALRT  computes  the  altitude  loss 
due  to  pilot  response  (reaction)  time.  GCASDIVE  computes  the  altitude  loss  as  a  result  of 
the  Dive/Climb  Angle  (DCA)  of  the  aircraft.  GCASROLL  computes  the  altitude  loss  due  to 
aircraft  roll  configuration.  These  three  subroutines,  GCASALRT,  GSASDIVE,  and 
GCASROLL,  in  addition  to  a  safety  buffer,  are  then  summed  to  make  the  total  predicted 
altitude  loss  (TALTLS).  The  resulting  total  is  compared  with  the  actual  aircraft  altitude  and 
a  warning  is  activated  if  the  actual  aircraft  altitude  is  less  than  the  algorithm  derived  total 
altitude  (MSGESTAT).  The  GCASLAND  subroutine  activates  the  TCHDWN  subroutine 
which  generates  a  warning  when  certain  conditions  are  encountered  during  approach  and 
landing  (explained  later).  No  computation  of  altitude  is  performed.  These  algorithms  are 
addressed  by  the  message  warning  subroutine  labeled  MSGESTAT.  See  Figure  1  for  the 
flow  diagram  of  the  Cubic  algorithm. 

The  GCASALRT  algorithm  uses  the  available  flight  path  angle,  vertical 
acceleration,  and  vertical  velocity  inputs  in  deriving  the  altitude  loss  due  to  pilot  response 

time  (Figure  2).  It  uses  the  equation:  DELTAZ  =  RESP  *  Vz  +  .5(AZ  *  RESP2).  If  A 
ALT  (altitude  change)  is  negative  (losing  altitude),  then  the  computed  GCASALRT  altitude 
is  used;  otherwise,  the  GCASALRT  altitude  is  set  to  zero.  The  GCASALRT  algorithm 
addresses  the  sub-algorithm,  PILTRESP.  PILTRESP  computes  the  predicted  pilot 
response  (RESP)  based  on  the  current  g-load  factor.  Predicted  PILTRESP  times  could 
vary  from  0.3  second  to  1.0  second. 

The  GCASDIVE  subroutine  uses  the  inputs  of  angle  of  attack,  bank  angle, 
barometric  pressure,  equivalent  airspeed,  flight  path  angle,  g-load,  true  airspeed,  vertical 
acceleration,  and  vertical  velocity  in  the  time-to-recover  and  time-to-target  g-load  equations 
to  generate  the  predicted  altitude  loss  due  to  dive  recovery  (Figure  3).  If  the  time-to-recover 
is  less  than  the  time-to-target  g-load,  then  altitude  loss  is  based  on  the  time-to-recover 
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equations;  otherwise,  the  altitude  is  calculated  based  on  the  time-to-target  g-load.  The 
GCASDIVE  algorithm  also  addresses  directly  or  indirectly  the  following  sub-algorithms: 
DCAEXTRP,  GCASKFIL,  GCASVEL,  and  GCASTERR  to  validate  the  reliability  of  the 
information  being  used  within  the  program.  Specifically,  DCAXTRP  computes  the 
extrapolated  incremental  Dive  Climb  Angle  (DCA)  which  is  added  to  the  actual  DCA  to 
allow  for  the  effects  of  terrain  slope.  For  response  times  not  greater  than  one  second,  the 
DCA  is  extrapolated  in  one  of  two  ways:  (1)  Using  vertical  velocity  (Vz)  and  change  in 
terrain,  and  (2)  Using  Vz  and  vertical  acceleration  (Az).  The  method  used  is  dependent  on 
the  dive  angle.  GCASKFIL,  a  subroutine  of  DCAXTRP,  provides  filtering  of  raw  radar 
altimeter  inputs.  The  Kalmen  filter  (GCASKFIL)  also  provides  above  ground  level  (AGL) 
reference  during  periods  when  the  radar  altitude  measurement  is  degraded  (e.g.,  radar 
altimeter  interference).  It  uses  the  flight  path  angle  (Gamma)  and  bank  angle  to  calculate 
the  extrapolation  time.  It  also  addresses  two  of  its  own  subroutines,  GCASTERR  and 
GCASVEL.  These  subroutines  provide  height  Above  Ground  Level  (AGL)  and  height 
above  Mean  Sea  Level  (MSL)  acceptance  used  to  ensure  radar  altimeter  accuracy. 

The  GCASROLL  subroutine  (Figure  4)  uses  aircraft  bank  angle,  equivalent 
airspeed,  roll  rate,  true  airspeed,  vertical  acceleration,  and  vertical  velocity  inputs  to 
compute  the  predicted  altitude  loss  due  to  roll  recovery.  Roll  angles  from  5°  to  45°  are 
covered  by  the  subroutine.  Roll  angles  less  than  5°  are  considered  to  have  no  effect  on 
altitude  loss.  Roll  angles  greater  than  45°  are  assumed  to  have  the  same  effect  as  the  45° 
roll  angle. 

The  resulting  altitudes  from  GCASALRT,  GCASDIVE,  and  GCASROLL  are  then 
summed  with  a  safety  buffer  altitude  calculated  in  the  subroutine  SAFTHR, 

(TALTLS  =  GCASALRT  +  GCASDIVE  +  GCASROLL  +  SAFTHR).  If  the  total 
computed  altitude  does  not  exceed  the  actual  aircraft  altitude  (AGL),  then  MSGESTAT  will 
not  generate  a  warning.  However,  should  the  total  computed  altitude  exceed  that  of  the 
aircraft,  then  a  GCAS  warning  will  be  triggered  by  the  MSGESTAT  algorithm. 

The  GCASLAND  subroutine  activates  the  TCHDWN  subroutine  when  the  aircraft 
altitude  is  less  than  1000  feet  (AGL),  and  two  of  the  following  three  conditions  are  met: 

(1)  Equivalent  airspeed  is  less  than  190  knots,  (2)  Flaps  are  greater  than  30%  deployed, 
and  (3)  Gear  are  more  than  50%  deployed.  If  the  above  conditions  are  not  met,  then  the 
MSGESTAT  subroutine  will  not  be  activated  by  the  GCASLAND  subroutine.  If  the 
conditions  above  are  satisfied,  then  the  TCHDWN  subroutine  is  activated  and  the  aircraft  is 
considered  to  be  prepared  for  the  approach  and  landing  phase  of  flight.  When  TCHDWN 
is  activated,  several  different  warnings  ("flaps,"  "gear,"  "glideslope,"  &  "pull-up")  may  be 
activated.  These  warnings  are  based  on  the  different  conditions  of  aircraft  altitude, 
glideslope  deviation,  equivalent  airspeed,  and  gear  and  flap  deployment. 

The  glideslope  warning  should  be  issued  when  the  following  four  conditions  are 
met:  (1)  Aircraft  altitude  exceeds  250  feet  (AGL),  but  is  less  than  1000  feet  AGL, 

(2)  Equivalent  airspeed  is  less  than  190  knots,  (3)  The  flaps  or  the  gear  or  both  are 
deployed  beyond  the  limits  described  previously,  and  (4)  Deviation  from  glideslope  is  less 
than  three  dots,  but  not  less  than  two  dots  (1  dot  =  .375  degrees).  If  the  altitude  is  less 
than  or  equal  to  100  feet  (AGL),  then  the  TCHDWN  subroutine  is  ignored  and  message 
warning  would  be  based  on  the  computed  altitude  loss  (TALTLS).  If  aircraft  altitude  is 
greater  than  100  feet  and  less  than  or  equal  to  250  feet,  or  the  altitude  is  between  250-1000 
feet  and  glideslope  deviation  is  not  less  than  three  dots,  then  the  message  warning  would  be 
based  on  a  computed  altitude  generated  by  a  modified  GCASDIVE  subroutine.  However, 
if  the  aircraft  is  between  250-1000  feet  (AGL)  and  glideslope  deviation  exceeds  two  dots 
but  is  less  than  three  dots,  then  a  glideslope  warning  would  be  issued. 
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A  gear  warning  is  generated  when:  (1)  Aircraft  altifi'  is  less  tlvi.  >00  feet  fAGL), 
(2)  Flaps  are  more  than  30%,  (3)  Equivalent  airspeed  is  ies$  than  190  knots,  and  (4)  Gear 
are  not  greater  than  50%  deployed.  A  flaps  warning  is  generated  when:  (1)  Aircraft  altitude 
is  less  than  500  feet  (AGL),  (2)  Gear  are  deployed  more  titan  50%,  (3)  Equivalent  airspeed 
is  less  than  190  knots,  and  (4)  Flaps  are  not  greater  than  30%.  The  glideslope  flaps,  and 
gear  warnings  in  the  TCHDWN  subroutine  are  automatically  generate!  when  the  above 
conditions  are  met.  No  computation  of  altitude  is  performed  for  these  warnings. 
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PHASE  I 


Method 


Pxocedure 

The  Cubic  GCAS  algorithm  was  initially  broken  down  into  its  different  subroutines 
to  determine  the  aircraft  inputs  needed  by  each  of  the  subroutines  to  compute  its  predicted 
altitude  loss.  As  seen  in  Figures  2-4,  the  GCASALRT,  GCASDIVE,  and  GCASROLL 
subroutines  require  many  of  the  same  inputs.  After  the  required  inputs  were  determined, 
the  subroutines  were  then  subjected  to  a  computation  analysis  requiring  each  subroutine  to 
compute  predicted  altitude  loss  given  the  different  aircraft  configurations.  A  Gould  Sel  87 
computer  connected  to  a  Nicolet  Zeta  824/836  CS  plotter  generated  the  predicted  altitude 
lost  graphs  for  each  of  the  main  subroutines.  These  altitude-lost  graphs  were  plotted  for 
varying  levels  of  gamma,  roll,  g-load,  altitude,  slope,  and  indicated  airspeed,  for  each  of 
the  sub-algorithms.  The  actual  independent  variables  varied  from  graph  to  graph  for  each 
of  the  GCASALRT,  GCASDIVE,  and  GCASROLL  subroutines  dependent  upon  the 
variables  used  by  the  subroutine  in  calculating  the  predicted  altitude  loss.  The  graphs  were 
then  analyzed  for  any  discrepancies  from  that  logically  expected  to  occur.  For  example,  if 
gamma  were  decreased  (dive  angle  increases),  one  would  logically  expect  the  predicted 
altitude  loss  to  increase. 


Results 


GCASALRT 

Figure  5  provides  an  example  of  the  predicted  altitude  loss  due  to  pilot  reaction  time 
(GCASALRT)  given  a  pilot  response  time  of  .65;  true  airspeeds  (TAS)  of  125, 175, 225, 
275,  and  325;  and  flight  path  angles  (Gamma)  ranging  from  0°  to  30°  nose  down.  As  we 
hypothesized,  predicted  altitude  loss  increased  as  both  airspeed  and  gamma  increased. 
Similar  trends  were  exhibited  for  pilot  response  times  of  .72,  .79,  .86,  .93,  and  1.00 
(Figures  6-10). 

GCASDIVE 

The  GCASDIVE  subroutine  was  evaluated  by  generating  various  predicted  altitude 
lost  graphs  based  on  a  Cg=22;  slopes  of  0,  6,  and  12;  barometric  altitudes  of  1000, 

10,000,  and  19,000;  g-loads  of  .5,  1.25,  and  2.0;  airspeeds  of  125,  175,  225,  275,  and 

325  knots;  and  gammas  ranging  from  0°-30°.  Predicted  altitude  loss  due  to  flight  path 
angle  (GCASDIVE)  graphs  are  shown  in  Figures  11-13.  As  seen  by  the  275  and  325  TAS 
data  lines,  there  is  some  interaction  of  the  higher  airspeeds  with  the  middle  airspeed  of  225 
between  0  and  13  degrees  gamma  for  the  lower  g-load  conditions.  The  g-load  variable  is 
the  g-load  of  the  aircraft  at  warning  initiation.  However,  this  interaction  disappears  at 
g-loads  of  1.25  and  2.00  (Figures  12  &  13).  Further  inspection  of  these  graphs  reveals 
gamma  and  airspeed  increases  resulted  in  increased  predicted  altitude  lost,  as  hypothesized. 
Additionally,  Figures  11,  12,  and  13  reveal  g-load  had  an  effect  on  GCASDIVE,  but  it  was 
interactive  in  nature,  as  the  predicted  altitude  loss  was  higher  for  both  the  low  (.5)  and  the 
high  (2.0)  g-load  conditions,  than  for  the  middle  (1.25)  g-load  condition. 
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To  investigate  this  anomaly,  time-to-target  load  subroutine  graphs  using  gamma, 
slope,  altitude,  and  g-load  as  independent  variables  were  developed.  The  algorithm  used  a 
target  g-load  of  1.5  g’s  and  then  calculated  the  time  to  reach  that  g-load.  As  expected,  an 
initial  g-load  of  2.0  resulted  in  a  negative  time  factor  as  the  target  g-load  had  already  been 
met  (Figure  14).  Figures  15  and  16  reveal  g-loads  of  0.5  and  1.25  did  result  in  different 
predicted  times  to  the  target  load;  however,  the  effect  of  airspeed  on  predicted  times  was 
random.  It  was  hypothesized  a  higher  airspeed  would  result  in  a  faster  time-to-target  load. 
Although  this  was  true  of  the  325  knot  true  airspeed,  the  next  quickest  time  curve  belonged 
to  the  125  knots,  not  the  275  knots,  as  expected.  Also,  a  comparison  of  Figure  15  with 
Figure  17  reveals  barometric  altitude  had  an  effect  on  time  to  target  Specifically,  the  order 
of  the  TAS  data  lines  changed  with  the  125  and  175  knots  showing  quicker  times  to  the 
target  load  than  the  other  airspeeds.  These  discrepancies  contributed  to  the  interactive  effect 
of  die  g-load  factor  on  the  GCASDIVE  predicted  altitude  loss. 

To  evaluate  the  effects  of  terrain  elevation  (at  the  time  of  warning)  on  the 
GCASDIVE  subroutine,  a  comparison  of  Figures  12, 18,  and  19  was  required.  These 
figures  represent  terrain  elevations  of  1000, 10,000,  and  19,000,  respectively.  The 
comparisons  between  the  figures  reveal  there  was  no  effect  on  predicted  altitude  loss  due  to 
terrain  elevation.  To  evaluate  the  effects  of  slope  on  GCASDIVE,  Figures  12, 20,  and  21 
were  compared.  These  figures  represent  slope  conditions  of  0, 6,  and  12  degrees.  No 
slope  effects  were  observed. 

To  further  investigate  the  reason  slope  had  no  effect  on  predicted  altitude  loss, 
predicted  altitude  lost  graphs  were  plotted  as  a  function  of  terrain  slope  (DCAEXTRP)  for 
airspeed  (125  &  225  knots),  slope  (0, 7,  &  14  degrees),  and  gamma  (0-30  degrees). 
Figures  22  &  23  indicate  increases  in  predicted  altitude  loss  as  both  airspeed  and  gamma 
increased,  as  expected.  No  effect  could  be  attributed  to  slope.  The  effect  was  so  minimal 
that  the  plotter  literally  typed  over  the  slope  identifiers  for  each  data  line,  as  evidenced  by 
the  superimposing  of  the  numerals  0, 7,  and  14.  Based  on  this  analysis,  it  became 
apparent  the  algorithm  failed  to  accurately  account  for  the  effects  of  slope. 

GCASROLL 

The  GCASROLL  predicted  altitude  lost  graphs  used  pressure  altitude  (1000, 
10,000,  19,000);  gamma  (-5,  -10,  -20,  -30);  airspeed  (125, 175, 225,  275,  325);  and  roll 
(0-90)  as  independent  variables.  An  inspection  of  Figure  25  indicates  both  airspeed  and 
roll  caused  increases  in  predicted  altitude  lost,  as  expected.  However,  the  effect  of  roll 
began  at  5  degrees  and  stabilized  at  a  roll  condition  of  45  degrees.  These  are  the  limits 
imposed  by  the  algorithm.  A  comparison  of  Figures  24  and  25  shows  that  as  gamma 
increased  so  did  the  predicted  altitude  lost.  By  reviewing  Figures  24  and  26  in  a  similar 
fashion,  we  find  pressure  altitude  had  no  effect  on  the  predicted  altitude  loss.  A  review  of 
the  remaining  graphs  (not  included  in  this  report )  paralleled  these  findings.  In  summary, 
gamma,  roll,  and  airspeed  all  had  the  expected  effects  on  predicted  altitude  loss  for  the 
GCASROLL  sub-algorithm. 


Discussion 

Phase  I  revealed  several  problem  areas  and  concerns  for  the  Cubic  GCAS 
algorithm.  These  problems  were:  (1)  The  effects  of  temperature  were  never  directly 
considered  by  the  algorithm.  (2)  The  algorithm  assumed  the  effects  due  to  Center  of 
gravity  (Cg)  were  zero.  (3)  The  DCAEXTRP  effects  of  slope  used  in  the  GCASDIVE 
subroutine  were  minimal.  (4)  The  time-to-target  load  plots  indicated  a  random  prediction  of 
times  relative  to  the  aircraft  airspeed.  Additionally,  the  algorithm  limited  the  target  load  to 
1.5  g;  whereas,  the  operational  limit  of  the  KC- 135  is  2.0. 
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According  to  the  KC-135  performance  manual,  T.0. 1C-135(K)R-1-1,  temperature 
directly  affects  engine  performance  (p.  1A2-2),  takeoff  and  climbout  performance 
(p.  1 A3-5),  climb  performance  (p.  1 A4-3),  descent  performance  (p.  1 A8-2),  and  approach 
and  landing  performance  (p.  1 A9-4).  Given  temperature  affects  the  aircraft  during  die 
critical  phases  of  flight  most  prone  to  CFIT  accidents  and  that  none  of  the  subroutines  uses 
temperature  as  an  input,  we  recommended  the  temperature  variable  be  considered  and  either 
directly  or  indirectly  incorporated  into  the  algorithm. 

In  a  similar  fashion,  gross  weight  affects  takeoff  and  climbout  performance 
(p.  1 A3-5),  descent  performance  (p.  1 A8-2),  and  approach  and  landing  performance 
(p.  1 A9-4).  Since  Cg  accounts  for  gross  weight  in  addition  to  the  aircraft  center  of 
moment,  the  need  for  the  algorithm  to  detect  and  account  for  Cg  effects  is  inherently  more 
important  By  accounting  for  the  effects  of  Cg,  the  algorithm  could  better  predict  the 
expected  altitude  loss  and,  therefore,  reduce  the  variability  of  its  prediction. 

As  seen  in  Figures  22  &  23,  the  DCAEXTRP  predicted  altitude  loss  due  to  the 
effects  of  terrain  slope  was  essentially  zero.  This  poses  a  potential  problem.  If  the 
algorithm  is  continually  calculating  for  a  no  slope  condition,  then  a  controlled  flight  into 
terrain  accident  would  be  much  more  likely  to  occur  under  conditions  of  increasing  slope. 
The  fact  the  algorithm  fails  to  account  for  the  effects  of  slope  renders  the  system  relatively 
useless  given  the  terrain  of  the  earth  is  generally  sloping  to  one  degree  or  another. 

Initial  indications  of  the  fourth  problem  area  suggested  the  variability  in  the 
prediction  of  the  time-to-target  across  true  airspeeds  resulted  in  an  interaction  of  G-load 
with  pressure  altitude.  Figures  15-17  indicate  this  unexpected  pattern  of  results  for 
time-to-target  load.  Accordingly,  the  predicted  altitude  loss  for  the  GCASDIVE  subroutine 
varied  in  a  similar  manner  for  the  different  g-load  conditions.  Since  Figures  15-17 
revealed  an  unexpected  pattern  of  results,  the  time-to-target  load  subroutine  needed  to  be 
reevaluated  by  Cubic. 

Phase  I  provided  an  initial  look  at  how  the  algorithm  was  designed  and  how  it 
should  work.  After  breaking  the  algorithm  down  into  its  main  subroutines,  we  were  able 
to  obtain  an  idea  of  how  well  the  algorithm  would  predict  and  under  what  conditions  or 
situations  there  may  be  a  problem.  The  problem  areas  listed  above  were  then  forwarded  to 
the  Cubic  Corporation  to  allow  them  to  incorporate  the  changes  they  deemed  necessary  into 
their  algorithm  prior  to  the  Phase  II  evaluation. 

Prior  to  Phase  II,  Cubic  only  addressed  one  of  the  four  concerns  found  during 
Phase  I.  Specifically,  Cubic  accounted  for  the  effects  of  temperature  through  the  use  of 
indicated,  as  well  as  true  airspeed.  Cubic  stated  the  problem  areas  concerning  the 
DCAEXTRP  effects  of  slope  and  the  time-to-target  load  plots  were  "OK"  and  did  not  need 
revision.  Cubic  also  stated  they  did  not  have  sufficient  information  to  incorporate  Cg  into 
their  algorithm  at  that  time.  Given  Cubic's  reply  and  SPO  direction,  the  Phase  II  portion  of 
our  evaluation  was  performed. 
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PHASE  II 


The  objective  of  the  Phase  II  evaluation  was  to  determine  the  GCAS  algorithm 
response  characteristics  as  a  function  of  five  independent  variables:  Terrain  slope,  terrain 
elevation,  indicated  airspeed,  aircraft  roll  angle,  and  flight  path  angle  (gamma).  The 
KC-135  simulator  was  placed  in  a  specific  configuration  based  on  the  independent 
variables  above.  Upon  release  of  the  simulator,  a  simple  robot  pilot  model  attempted  to 
stabilize  the  simulator  at  one  "G"  while  maintaining  the  given  run  conditions  until  a  pull-up 
warning  was  initiated.  Following  a  predetermined  delay,  based  on  the  algorithm's 
calculations  for  predicted  pilot  reaction  time,  the  pull-up  maneuver  was  then  performed  by 
the  pilot  model  based  on  the  following  criteria:  Roll  and  pull  simultaneously  to  a  maximum 
of  three  Gs.  This  condition  was  maintained  until  the  flight  path  angle  of  the  aircraft  became 
greater  than  that  of  the  terrain. 

Phase  II  was  run  in  two  separate  parts  using  the  robot  pilot  model.  Each  part 
followed  the  described  methodology  below.  Part  1  was  the  initial  analysis  performed  on 
the  revised  Cubic  algorithm  from  Phase  I.  Cubic  then  responded  to  the  problems  areas 
found  in  Part  1.  Part  2  was  an  evaluation  of  the  revised  Cubic  algorithm,  which  included 
the  changes  Cubic  deemed  necessary  to  correct  the  deficiencies  found  during 
Phase  II  -  Part  1.  Each  pan  will  be  discussed  in  detail. 


Method 


Apparatus 

Facility.  The  study  was  conducted  at  the  Crew  Station  Evaluation  Facility 
(CSEF),  which  is  a  U.S.  Air  Force  simulation  facility  that  belongs  to  the  Aeronautical 
Systems  Division  (ASD)  of  Air  Force  Systems  Command.  CSEF  government  personnel 
are  assigned  by  the  Crew  Systems  Division  (ASD/ENEC).  The  facility  is  used  to  perform 
human  engineering  experiments  in  support  of  a  variety  of  System  Program  Offices. 

Simulator.  The  KC-135  simulator,  shown  in  Figures  27  and  28,  rested  on  a 
three  degrec-of-frcedom  motion  platform,  and  included  such  major  components  as  the 
control  loading  assemblies,  seats,  yokes,  and  visual  windows.  The  simulator  was 
equipped  with  two  wide  angle  collimating  windows  that  provided  a  panoramic  outside 
scene  capable  of  supporting  the  CSEF  Night  Visual  System  (NVS).  A  Digital  Equipment 
Corporation  (DEC)  PDP  11/35  computer  used  one  of  a  number  of  databases  to  generate 
sets  of  lights,  simulating  various  night  visual  scenes,  for  the  NVS.  This  provided  the  pilot 
with  a  visual  capability  used  in  Phases  HI  and  IV  of  our  study.  The  KC-135  simulator 
contained  all  the  instrumentation  found  in  the  actual  cockpit  for  both  the  pilot  and  copilot 
positions.  The  software  package  contained  all  flight,  engine,  atmosphere,  weights  and 
balances  modules;  a  dictionary  of  all  KC-135  data  variables;  and  several  other  specific 
commons  and  data  pools  for  the  KC- 1 35C  model  aircraft 

Computer  Complex.  The  simulator  was  connected  to  a  series  of  large  and  small 
computer  systems.  This  computer  complex  included  five  Gould  series  32/7780,  one  Gould 
concept  32/8780,  two  PDP  1 1/34,  three  PDP  1 1/35,  and  several  Silicon  Graphics  Iris 
Work  Stations. 
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Design 

The  objective  of  the  Phase  n  evaluation  was  to  determine  the  GCAS  algorithm 
response  characteristics  as  a  function  of  rive  independent  variables:  Terrain  slope,  terrain 
elevation,  indicated  airspeed,  aircraft  roll  angle,  and  flight  path  angle  (gamma).  Terrain 
slope  was  the  slope  of  the  terrain  at  warning  initiation;  whereas,  terrain  elevation  was  the 
height  of  the  terrain  (MSL)  at  the  beginning  of  each  simulation  trial.  The  remaining  three 
variables  were  recorded  at  the  time  of  warning  initiation.  Table  1  lists  the  different  levels 
for  each  of  the  independent  variables. 


Table  1.  Phase  II  independent  variables. 


ROLL 

GAMMA 

IAS 

SLOPE 

ELEVATION 

0 

-5 

225 

0 

1000 

15 

-10 

275 

7 

10000 

30 

-15 

325 

14 

45 

-20 

The  actual  condition  of  the  aircraft  and  the  predicted  altitude  loss  by  each  of  the 
subroutines  were  recorded  for  each  of  the  robot  runs.  Three  dependent  measures  of 
interest  were  also  recorded  for  subsequent  data  analysis.  These  were  maximum  g's, 
minimum  clearance  altitude,  and  total  altitude  lost  Maximum  g's  represented  the  highest 
instantaneous  g-forcc  placed  on  the  aircraft,  and  acted  as  our  criterion  for  accepting  or 
rejecting  a  given  robot  run.  If  a  run  exceeded  3.0  g's,  then  the  run  was  discarded  and  a 
new  run  performed.  This  value  was  chosen  as  it  represented  the  structural  limit  of  the 
aircraft  (3.0). 

Total  altitude  lost  allowed  the  comparison  of  the  actual  total  altitude  loss  with  the 
algorithm’s  predicted  total  altitude  loss.  The  minimum  clearance  altitude,  defined  as  the 
minimum  distance  between  the  aircraft  and  the  ground  (feet-AGL)  during  the  aircraft's 
recovery,  was  the  primary  dependent  variable  of  interest  This  variahle  provided  the 
experimenter  with  the  information  needed  to  determine  if  the  algorithm  had  provided 
adequate  ground  clearance.  The  overall  experimental  design  was  comprised  of  a  single  run 
per  condition  for  a  total  of  288  trials  (4  Roll  x  4  Gamma  x  3  IAS  x  3  Slope  x  2  Elevation). 
All  runs  were  successfully  completed. 

Procedure 

A  set-up  control  interface  program  developed  to  simplify  user-computer  interaction 
allowed  the  experimenter  to  monitor  real-time  characteristics  of  the  simulator  as  it  flew  each 
configuration.  Table  2  presents  an  example  of  the  computer  program  page  die  experimenter 
used  to  manipulate  a  number  of  variables  related  to  the  simulator,  the  terrain  it  flew  over, 
and  the  response  characteristics  of  the  pilot  model.  The  actual  pilot  model  was  fashioned 
after  die  response  patterns  of  two  individual  operational  KC-135  pilots.  These  data  had 
been  gathered  through  an  earlier  pilot  study. 
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7.  ALTITUDE 
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PRESS  'R'  TO  RUN 


Upon  releasing  the  aircraft,  the  experimenter  was  presented  with  a  new  data  page 
that  allowed  the  monitoring  of  real  time  simulator  performance  characteristics  (Table  3). 

The  pilot  model  attempted  to  stabilize  and  maintain  the  simulator  at  one  G,  until  the  warning 
signal  was  initiated.  The  response  delay  of  the  robot  pilot  model  was  configured  so  that 
the  pull-up  maneuver  was  delayed  for  approximately  the  same  amount  of  time  as  that 
calculated  for  the  pilot  response  time  by  the  algorithm's  PILTRESP  subroutine.  Rolling 
out  and  pulling  back  were  executed  simultaneously  with  a  maximum  of  two  and  a  half 
G's.  The  recovery  procedure  continued  until  a  positive  radar  altitude  rate  of  climb  was 
established.  At  run  completion,  the  experimenter  could  plot  the  course  and  the  performance 
characteristics  exhibited  throughout  the  pull-up  maneuver  (Figure  29).  This  allowed 
immediate  determination  of  the  validity  of  the  run. 
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275. 

14. 

1000. 

5000. 

2. 


A.  INCREMENT  RUN  :  ON 

B.  DATA  COL  :  OFF 


Table  3.  An  example  of  the  data  display  naye  available  to  the  experimenter. 
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Figure  29  displays  the  initial  run  conditions  at  the  top  of  the  page  as  well  as  in  the 
first  six  lines  along  the  right  side  of  the  graph.  The  next  four  lines  along  the  side  of  the 
graph  were  the  algorithm's  predicted  altitude  loss  for  each  of  the  subroutines, 


GCASALRT,  GCASROLL,  and  GCASDIVE,  in  addition  to  the  algorithm's  calculated 
safety  buffer.  These  were  then  summed  to  obtain  the  total  predicted  altitude  loss,  ALT 
LOSS.  The  reaction  time  listed  is  that  used  by  the  algorithm.  The  roll,  gamma,  altitude, 
airspeed,  and  vertical  velocity  (VZ)  found  in  the  third  group  of  variables,  were  the  actual 
conditions  of  the  aircraft  simulator  at  warning  initiation.  Maximum  G's  (MAX  Gs)  and 
actual  total  altitude  loss  (ALT  LOSS)  were  the  maximum  values  obtained  during  aircraft 
recovery.  Minimum  clearance  (MIN  ALT)  represents  the  minimum  distance  between  the 
aircraft  and  the  terrain  that  occurred  during  the  GCAS  recovery.  The  roll  line  and  Gs  line 
represent  the  real  time  roll  and  g-load  values  of  the  aircraft  during  its  flight.  The  x-axis  is 
the  total  running  time  since  the  run  began.  The  y-axis  is  the  altitude  of  the  aircraft  in  feet 
(MSL).  The  flight  of  the  aircraft  is  represented  by  the  curved  upper  line  beginning  at  the 
maximum  value  of  the  y-axis.  The  ground  level  is  represented  by  the  straight  line  that 
begins  at  a  terrain  elevation  of  1000  feet. 

Several  hypotheses  were  generated  based  on  earlier  discussions  with  pilots  and 
previous  GCAS  work  (Hassoun,  Ward,  Bamaba,  &  McCarthy,  1989).  It  was 
hypothesized  that  increases  in  gamma,  roll,  airspeed,  terrain  slope,  and  terrain  elevation 
would  result  in  increased  minimum  clearances.  This  is  based  on  the  finding  that  as 
downward  vertical  velocity  increased,  pilots  felt  a  corresponding  increase  in  minimum 
clearance  would  be  necessary  to  provide  a  safe  and  comfortable  recovery.  Gamma  and 
indicated  airspeed  directly  affect  downward  vertical  velocity.  Terrain  elevation  indirectly 
affects  vertical  velocity  by  providing  decreased  air  densities  as  the  elevation  or  mean  sea 
level  increases.  The  hypothesized  roll  effect  was  based  on  pilot  comments  that  they  felt  the 
increased  workload,  roll  recovery  in  addition  to  dive  recovery,  would  dictate  a  desired 
higher  minimum  clearance.  Increased  terrain  slope  was  hypothesized  to  increase  minimum 
clearance  altitudes,  as  rising  terrain  would  cause  additional  stress  on  the  pilot  and  must, 
therefore,  be  compensated  by  higher  minimum  clearance  altitudes. 

Part  1  Results 

No  formal  statistical  analyzes  were  performed  on  the  data  since  only  one  subject 
(the  robot  aircraft)  existed.  However,  the  data  from  all  288  runs  were  sorted  by  indicated 
airspeed,  terrain  slope,  and  terrain  elevation,  and  graphed  for  minimum  clearance  as  a 
function  of  flight  path  angle  (gamma)  for  all  the  roll  angle  conditions.  A  comparison  of 
Figures  30-47  with  each  other  provided  the  information  needed  to  determine  the 
algorithm's  ability  to  provide  adequate  minimum  clearance. 

lerrain-JEkyatton  Effects 

To  determine  the  effects  due  to  terrain,  the  1000-foot  terrain  elevation  had  to  be 
compared  with  the  same  indicated  airspeed  and  terrain  slope  graphs  for  a  terrain  elevation 
of  10,000  feet  This  required  the  following  comparisons:  Figures  30  &  39,  Figures  31  & 
40,  Figures  32  &  41,  etc.  The  result  of  such  comparisons  indicated  the  elevation  of  the 
terrain  had  a  direct  effect  on  the  minimum  clearance  altitude  of  the  aircraft.  The  effect  was 
in  the  hypothesized  direction,  minimum  clearance  increased  as  terrain  elevation  increased. 
The  effect  also  increase^  more  dramatically  as  the  flight  path  angle  of  the  aircraft  increased. 
Since  the  effect  was  con  ant  across  graph  comparisons  and  acted  to  increase  the  minimum 
clearance  provided,  we  felt  the  terrain  elevation  variable  was  being  adequately  considered 
by  the  algorithm.  Given  elevation  effects  were  adequately  considered  and  report  brevity  is 
highly  desired,  further  discussion  of  the  graph  comparisons  will  focus  on  the  1000-foot 
terrain  elevation  graphs  (Figures  30-38);  although,  similar  comparisons  by  altitude  could  be 
and  were  made  during  the  actual  analysis. 


39 


Tail* 


GAMMA 


■10*1 


GAMMA 


UJW<0*<ZUW  <JH«h5QW 


45 


OJW<od<ZOW  <JH«H5QW 


50 


§ 

v> 

o 

T 

1 

II 

-j 

o 

z 

06 

o 

0 

p 

< 

> 

u 

a 

o 

UJ 

0£ 

o 

vn 

II 

1 

II 

w 

s 

8 

So 

a 

<n 

<N 

1 

m 

□ 

II 

9 

V3 

B 

< 

* 

GAMMA 


GAMMA 


GAMMA 


GAMMA 


GAMMA 


GAMMA 


GAMMA 


Airspeed  Effects 


Comparisons  of  Figures  30-32,  Figures  33-35,  and  Figures  36-38  provided  the 
analysis  for  indicated  airspeed  effects.  In  a  similar  fashion  to  terrain  elevation  effects, 
indicated  airspeed  increases  resulted  in  increased  minimum  clearances.  The  effect  was  also 
enhanced  for  the  higher  gammas.  This  was  anticipated  since  increased  minimum  clearances 
were  desired  by  pilots  for  increased  flight  path  angles  and  increased  airspeeds.  The 
algorithm  adequately  addressed  the  effects  of  indicated  airspeed. 

Roll  Angle  Effects 

To  determine  the  effects  of  roll  angle,  each  graph  was  inspected  for  the  different 
levels  of  roll.  Each  line  represented  a  given  roll  condition.  It  was  anticipated  that  minimum 
clearance  would  increase  as  the  roll  angle  increased.  A  review  of  Figure  30  supported  this 
hypothesis.  Although  there  was  some  interaction  of  the  roll  lines  across  the  figures, 
increased  roll  did  generally  cause  increased  minimum  clearances.  The  interaction  was 
typically  caused  by  the  zero  roll  condition  and  may  be  due  to  the  fact  that  the  algoritlim  does 
not  compute  a  predicted  altitude  loss  due  to  roll,  until  a  roll  greater  than  five  degrees  exists. 

Flight  Path  Angle  Effects 

The  effects  of  flight  path  angle  can  be  determined  by  proceeding  along  each  of  the 
roll  lines  on  each  of  the  graphs.  A  review  of  Figure  30  indicates  gamma  generally  had  the 
hypothesized  effect  on  minimum  clearance.  Specifically,  increased  gammas  resulted  in 
increased  minimum  clearance  altitudes.  This  was  generally  true  of  tile  remaining  figures. 

Terrain  Slope  Effects 

The  effects  of  terrain  slope  on  minimum  clearance  altitude  can  be  readily  seen  by 
comparing  Figures  30,  33,  and  36;  Figures  31, 34,  and  37;  and  Figures  32, 35,  and  38. 

By  doing  so,  one  observes  terrain  slope  had  a  great  effect  on  the  minimum  clearance 
altitude.  As  terrain  slope  increased,  minimum  clearance  decreased  rapidly,  so  rapidly  that 
terrain  slopes  of  7  and  14  degrees  yielded  minimum  clearances  well  below  ground  level. 
The  algorithm  did  not  adequately  accounted  for  the  effects  of  terrain  slope. 

Part  1  Discussion 

The  above  analysis  revealed  a  major  discrepancy  in  the  algorithm's  ability  to 
provide  adequate  ground  clearance  under  rising  terrain.  This  inability  to  provide  safe 
minimum  clearances  was  further  aggravated  as  terrain  slope  and  flight  path  angles  became 
larger.  Figures  38  and  47  clearly  show  high  terrain  was  improperly  accounted  for  by  the 
GCAS  algorithm.  A  review  of  Table  4  furthers  supports  this  finding.  Table  4  contains  the 
mean  minimum  clearance  altitude  of  a  given  independent  variable  for  both  Part  1  and 
Part  2.  These  findings  again  reinforce  the  Phase  I  conclusion  that  the  effects  of  slope  were 
not  adequately  considered  by  the  DCAEXTRP  sub-algorithm. 

As  seen  in  Table  4,  terrain  elevation  did  provide  the  expected  pattern  of  results  in 
minimum  clearances.  Consequently,  terrain  elevation  was  held  constant  and  was  not  an 
independent  variable  dui  ing  Phase  II-Part  2.  Gamma  was  the  only  other  variable  that 
provided  the  expected  pattern  of  means,  but  due  to  gamma's  interaction  with  terrain  slope, 
discussed  previously,  the  gamma  variable  was  retained  as  a  variable  of  interest  in  Part  2. 
Because  they  failed  to  provide  the  expected  pattern  of  mean  minimum  clearances  (Table  4), 
both  the  airspeed  and  roll  angle  variables  were  retained  forevaluation  during  Part  2. 
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30 

-59 

534 

613 

386 

45 

-50 

681 

675 

332 

Right 

5 

-233 

302 

284 

230 

Path 

10 

-214 

388 

422 

280 

Angle 

15 

-103 

535 

589 

209 

(Dive) 

20 

746 

258 

999 

294 

Terrain 

0 

440 

365 

440 

350 

Slope 

7 

-23 

292 

577 

369 

14 

-635 

349 

714 

348 

Cubic  attempted  to  correct  the  problem  by  revising  the  algorithm's  calculation  of  the 
g-onset .  Cubic  did  this  by  converting  the  g-onset  calculation  from  a  linear  model  that 
addressed  a  look-up  table  into  a  fully  linear  model  using  the  least  squares  method  with  the 
target  load  held  constant.  This  affected  the  time-to-target  calculations  discussed  earlier 
during  Phase  I.  As  will  be  shown  in  Part  2,  this  resulted  in  a  great  improvement  in  the 
performance  of  the  Cubic  GCAS  algorithm. 


Part  2  Results 

Prior  to  further  evaluation,  the  Cubic  algorithm  was  revised  to  account  for  the 
effects  of  slope.  Part  2  evaluated  the  revised  algorithm’s  ability  to  adequately  provide 
ground  clearance.  Since  the  algorithm  was  already  adequately  accounting  for  terrain 
elevation  effects,  terrain  elevation  was  not  evaluated  during  Part  2.  Consequently,  50% 
fewer  trials  were  run  for  a  total  of  144  trials.  Two  additional  variables,  Cg  and  Gross 
weight,  were  evaluated  during  Part  2  for  an  additional  48  trials.  As  in  Part  1,  the  data  were 
sorted  by  indicated  airspeed  and  terrain  slope,  and  graphed  for  minimum  clearance  as  a 
function  of  flight  path  angle  (gamma)  for  all  roll  angle  conditions. 
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Airspeed  effects  were  evaluated  by  comparing  Figures  48-50,  Figures  51-53,  and 
Figures  54-56.  Increased  minimum  clearance  altitudes  were  associated  with  increased 
airspeed.  No  significant  deviations  from  this  trend  were  noted.  These  comparisons  indicate 
the  effects  of  airspeed  were  adequately  accounted  for  by  the  revised  Cubic  algorithm. 

Roll  Angle  Effects 

In  a  similar  manner  to  the  gamma  effects,  increased  roll  angles  resulted  in 
corresponding  increases  in  the  minimum  clearance  altitudes  for  Figures  48-54.  Only  under 
the  high  airspeed-high  slope  conditions  did  an  interaction  of  the  roll  lines  occur,  as 
evidenced  in  Figures  55  and  56.  Accordingly,  the  revised  Cubic  algorithm  adequately 
accounted  for  the  effects  of  roll  angle. 

Flight  Path  Angle  Effects 

Gamma  effects  were  as  predicted.  Increased  dive  angles  resulted  in  increased 
minimum  clearance  altitudes  (see  Figures  48-54).  Only  Figures  55  and  56  show  any 
deviations  away  from  this  pattern.  These  again  were  found  under  the  high  terrain  slope 
conditions.  Comparison  between  Figures  30-38  with  Figures  48-54  reveal  similar  results, 
with  increased  minimum  clearances  for  the  higher  gamma  conditions  being  generated  by  the 
revised  algorithm.  Given  this  adjustment,  the  revised  algorithm  appears  to  better  account 
for  the  effects  of  flight  path  angle. 

Terrain  Slone  Effects 

To  examine  the  effects  of  slope,  Figures  48, 51,  and  54;  Figures  49, 52,  and  55; 
and  Figures  50, 53,  and  56  had  to  be  compared.  Generally  speaking,  increased  terrain 
slope  resulted  in  increased  minimum  clearance  altitude.  A  comparison  of  Figures  30-38 
with  Figures  48-56  indicate  the  problem  originally  associated  with  terrain  slope  had  been 
corrected.  Specifically,  the  revised  algorithm  provided  increased  minimum  clearances  and 
resulted  in  positive  ground  clearances  for  all  data  runs.  The  former  algorithm  had  impacted 
the  ground  numerous  times. 

Gross  Weight  and  Center  of  Gravity  Effects 


To  evaluate  the  effects  of  gross  weight  (GW)  and  center  of  gravity  (Cg),  48  more 
data  runs  were  performed.  The  gross  weight  runs  were  performed  with  a  constant  Cg  of 
24.  Cg  runs  were  performed  at  a  constant  GW  of  270  thousand  pounds.  These  runs  were 
then  graphed  as  previously  described,  but  with  the  added  variable  of  GW  or  Cg.  A 
comparative  analysis  of  Figures  57  &  58  revealed  100  foot  differences  in  minimum 
clearance  altitudes  as  a  result  of  a  gross  weight  change.  This  effect  was  considered 
minimal  after  an  inspection  of  Figures  59-61  revealed  that  changes  in  the  center  of  gravity 
consistently  resulted  in  minimum  clearance  altitude  changes  of  more  than  two  hunched  feet. 
Given  the  effects  Cg  plays  on  minimum  clearance  altitudes,  the  algorithm  should  address 
the  Cg  variable.  The  algorithm  as  currently  written  fails  to  consider  Cg  and  assumes  the 
effects  of  Cg  are  negligible. 
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GAMMA 


Phase  II  Discussion 


The  Phase  n  Evaluation  employed  a  pilot  model  to  test  the  ability  and  consistency  in 
predicting  required  altitude  as  the  algorithm  changed  paths.  During  Part  1  of  our  Phase  n 
evaluation,  we  concluded  the  algorithm  did  not  adequately  account  for  the  effects  of  terrain 
slope.  This  failure  was  evidenced  by  the  numerous  ground  collisions  experienced  by  the 
robot  pilot  model  during  its  recovery  procedure.  Generally,  the  results  found  during 
Part  1  confirmed  the  findings  from  Phase  I,  the  algorithm's  inability  to  accurately  account 
for  the  effects  of  terrain  slope.  These  findings  were  then  forwarded  to  Cubic  and  the 
revised  algorithm  was  evaluated  in  Part  2. 

It  was  determined  in  Part  2  that  the  algorithm  had  improved  in  its  ability  to  provide 
adequate  minimum  clearance.  This  became  quite  apparent  when  no  ground  impacts 
occurred  during  the  Part  2  data  runs.  Additionally,  trend  analysis  and  graph  comparisons 
revealed  that  all  of  the  gamma,  roll,  airspeed,  and  terrain  slope  variables  caused  the 
expected  effects  in  minimum  clearance  altitudes.  By  reviewing  the  mean  minimum 
clearances  for  Part  2  (Table  4),  one  finds  the  hypothesized  pattern  of  results  for  all  four 
independent  variables.  Terrain  slope  results  were  greatly  improved,  as  evidenced  by  a 
complete  reversal  in  the  order  of  the  means  from  Part  1  (original  algorithm)  to  Part  2 
(revised  algorithm).  Additionally,  comparisons  of  the  standard  deviations  from  Part  1  with 
the  Part  2  standard  deviations  reveal  the  variability  in  the  minimum  clearances  for  each  of 
the  independent  variables  decreased.  Only  the  terrain  slope  standard  deviations  remained 
relatively  constant.  These  findings  indicated  the  revised  algorithm  (Part  2)  had  been  greatly 
improved. 

One  concern  did  arise  from  the  Part  2  portion  of  our  Phase  II  evaluation.  The 
effects  of  Cg  were  not  accounted  for  by  the  algorithm.  Our  findings  indicated  Cg  did  have 
an  effect  on  the  minimum  clearance,  and  consequently,  should  be  considered  in  the  design 
of  the  GCAS  algorithm.  This  finding  also  supported  our  Phase  I  finding  that  the  effects  of 
Cg  should  be  considered.  However,  a  potential  problem  does  exist  with  the  incorporation 
of  Cg.  At  present,  the  KC-135  aircraft  does  not  have  a  sensor  that  detects  Cg.  Therefore, 
some  sensor  would  need  to  be  installed  in  the  KC-135  aircraft  to  allow  the  algorithm  to 
account  for  the  effects  of  Cg.  This  might  prove  to  be  cost  prohibitive.  Prior  to  the  Phase 
HI  and  Phase  IV  evaluations,  Cubic  made  no  further  corrections  to  their  algorithm. 
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PHASE  III 


The  objective  of  the  third  phase  of  the  evaluation  was  to  introduce  the  pilot  factor 
and  replicate  a  subset  of  the  runs  flown  by  the  robot  pilot  model  throughout  Phase  II,  with 
an  increased  emphasis  on  standard  flying  configurations.  Phase  III  allowed  C/EC/KC-135 
pilots  to  fly  a  GCAS  equipped  KC-135  simulator  in  order  to  evaluate  and  critique  the  most 
current  version  of  the  GCAS  algorithm.  The  critique,  as  well  as  the  performance  and 
subjective  results,  was  in  turn  shared  with  the  SPO  and  the  designers  (Cubic  Defense 
Systems)  for  modification  of  the  algorithm.  Phase  M  was  also  broken  into  two  parts. 
During  Part  1,  pilots  flew  a  subset  of  the  dive  configuration  runs  performed  during 
Phase  II.  For  Part  2,  pilots  flew  eight  1LS  approaches  and  the  resulting  data  were  analyzed 
to  determine  the  predictive  accuracy  of  the  GCASLAND  subroutine. 

Method 


Subjects 

A  total  of  nine  pilots  (eight  males  and  one  female)  rated  in  the  C/EC/KC-135  were 
used.  Due  to  the  state  of  world  affairs  at  the  time,  only  three  of  the  subjects  were 
operational  pilots  current  in  the  aircraft.  The  remaining  six  were  chosen  from  a  list  of 
volunteers  from  the  various  offices  located  at  Wright-Patterson  AFB,  Ohio.  The  pilot's 
were  required  to  have  been  an  aircraft  commander  in  the  KC-1 35.  The  resulting  personal 
data  indicated  the  subject  pool  was  relatively  older  (x=31.98),  more  experienced  (average 
total  flight  hours=2819  and  average  total  KC-135  flight  hours=1739),  but  less  current  in 
the  aircraft  (average  time  in  months  since  last  flight  =25)  than  the  current  operational  force. 

Apparatus 

Facility.  Refer  to  the  Phase  II  Apparatus  section  for  a  description. 

Computer  Complex.  Refer  to  the  Phase  II  Apparatus  section  for  a  description. 

Simulator.  Refer  to  the  Phase  II  Apparatus  section  for  a  description  of  the 
KC-135  simulator.  Additionally,  a  console  located  behind  the  pilot's  seat  allowed  the 
experimenter  to  remain  in  the  cockpit  during  training  to  provide  the  pilot  with  needed 
hands-on  demonstrations  and  explanations.  The  decision  height  lights  and  warning  horn 
were  disabled  to  keep  the  pilot  from  anticipating  a  possible  GCAS  warning. 

Experimenter's  Console.  The  experimenter's  console  was  located 
approximately  10  feet  away  for  the  simulator.  It  included  a  complete  intercom  system,  with 
communication  to  and  from  the  pilot  inside  the  simulator.  The  console's  displays 
duplicated  the  simulator's  instruments  and  displays,  and  were  used  to  monitor  the  pilot  and 
aircraft  performance.  Furthermore,  the  console's  controls  permitted  the  experimenter  to 
start,  stop,  and  reset  the  simulation  at  any  time. 

Voice  Message  Unit  Mechanization.  One  warning  and  three  caution  voice 
messages  were  presented  to  the  pilot's  headset  through  the  intercom  channel.  The 
messages  were  "Pull-up,"  "Flaps,"  "Gear,"  and  "Glideslope."  The  pilots  were  allowed  to 
set  the  volume  level  of  the  interphone  as  they  saw  fit.  The  warning  message  was 
mechanized  in  accordance  with  the  Cubic  algorithm.  This  dictated  the  warning  be 
presented  once,  then  iniiibited  for  a  five-second  period.  At  the  end  of  the  five-second 
period,  the  warning  would  only  be  repeated  if  the  aircraft  was  still  in  a  warning  condition. 
The  "Pull-up  "  message  was  presented  in  pairs  ("Pull-up/Pull-up")  with  an  inter-message 


76 


interval  of  500  milliseconds.  The  "Flaps,"  "Gear,"  and  "Glideslope"  messages  were 
presented  once  each  for  the  given  condition  and  then  inhibited  for  the  five-second  period. 

Audio  Systems.  The  voice  messages  were  recorded  on  an  Amiga  micro 
computer  by  a  female  employee  of  the  CSEF.  The  employee,  who  had  a  distinctive  and 
mature  mid-western  voice,  presented  the  messages  in  a  formal  and  impersonal  manner. 

The  Amiga  used  a  high  speed  voice  digitizer  (Future  Sounds),  with  a  sampling  rate  of 
10,000  samples  per  second,  to  convert  the  messages  from  analog  to  digital  format  The 
Amiga  was,  thereafter,  connected  to  the  main  frame  computers  using  an  RS-232  interface, 
and  transmitted  the  messages  to  the  pilot's  headset  (an  ASTROCOM  model  number  20680 
with  MX-2508/A/C  pads)  through  the  intercom  channel. 

Visual  Warning  Signal.  A  flashing  red  light  placed  fifteen  degrees  right  of  the 
center  field  of  view  and  just  to  the  left  of  the  engine  and  fuel  instrument  panels  provided  the 
visual  warning  stimulus.  The  word  "Altitude"  was  etched  in  black  lettering  on  a  red 
background.  This  type  of  light  and  nomenclature  was  chosen  based  on  the  results  of  an 
earlier  GCAS  questionnaire  (Rueb  &  Hassoun,  1990). 

Part  1  •  Dive  Configuration 

Design 

Phase  III  of  the  evaluation  was  designed  to  compare  C/EC/KC-135  pilots' 
performance  and  subjective  data  as  a  function  of  four  independent  variables.  These 
variables  were:  (1)  terrain  slope,  (2)  indicated  airspeed,  (3)  roll  angle  (both  left  and  right  to 
avoid  response  biases),  and  (4)  flight  path  angle  (gamma).  Table  5  presents  the  levels  for 
each  of  the  independent  variables.  Tlie  four  independent  variables  were  recorded  at  the 
time  of  warning  initiation. 
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The  actual  condition  of  the  aircraft  and  the  total  altitude  loss  estimated  by  each  of  the 
subroutines  were  recorded  for  each  of  the  data  runs.  Three  dependent  measures  of  interest 
were  also  recorded  for  subsequent  data  analysis.  These  were  maximum  G's,  minimum 
clearance  altitude,  and  total  altitude  lost  Maximum  G's  represented  the  highest 
instantaneous  g-force  placed  on  the  aircraft,  and  acted  as  one  of  our  criteria  for  accepting  or 
rejecting  a  given  run.  If  a  run  exceeded  2.5  g's,  then  the  run  was  discarded  and  a  new  run 
performed.  This  value  was  chosen  as  it  represented  the  midpoint  between  the  operational 
limit  (2.0)  and  the  structural  limit  (3.0)  of  the  aircraft. 

Total  altitude  lost  allowed  the  comparison  of  the  actual  total  altitude  loss  with  the 
algorithm’s  predicted  total  altitude  loss.  The  minimum  clearance  altitude,  defined  as  the 
minimum  distance  between  the  aircraft  and  the  ground  (feet-AGL)  during  the  aircraft's 
recovery,  was  the  primary  dependent  variable  of  interest.  This  variable  provided  the 
experimenter  with  the  information  needed  to  determine  if  the  algorithm  had  provided 
adequate  ground  clearance.  The  overall  experimental  design  was  a  repeated  measures 
design  for  which  each  pilot  was  required  to  fly  each  of  the  24  different  run  conditions 
(2  Roll  x  3  Gamma  x  2  IAS  x  2  Slope).  All  mns  were  successfully  completed. 


Procedure 


The  pilot  was  given  a  standardized  briefing  explaining  the  background  for  the 
study,  the  simulator's  capabilities  and  peculiarities,  and  the  particular  flight  profiles  they 
would  be  required  to  fly.  Upon  completion  of  the  briefing,  the  pilot  was  provided 
instruction  on  the  simulator  GCAS  warning  and  on  the  various  locations  of  the  aircraft 
controls.  The  pilot  was  then  allowed  to  fly  the  aircraft  simulator  until  they  felt  comfortable 
with  their  ability  to  fly  the  simulator  proficiently.  In  all  cases,  the  pilot  felt  comfortable 
enough  with  his/her  ability  to  fly  the  simulator  proficiently  within  two  hours  of  beginning 
the  training  period.  At  that  time,  the  pilot  began  the  Part  1  portion  of  the  Phase  in 
evaluation. 

Pan  I  required  the  pilot  to  perform  a  subset  (See  Table  5)  of  the  dive  configuration 
runs  performed  during  Phase  II  of  our  evaluation.  This  portion  of  the  evaluation  simulated 
flying  in  the  weather  without  an  outside  window  visual  scene.  In  an  attempt  to  decrease 
pilot  reaction  time  performance  biases  (i.e.,  possibility  of  the  pilot  accurately  anticipating 
the  warning),  the  simulator  was  initially  frozen  at  an  altitude  between  1000-5000  feet  above 
the  estimated  warning  altitude.  Prior  to  the  release  of  the  KC-1 35  simulator,  the  pilot  was 
given  the  desired  aircraft  parameter;  (Roll  angle,  IAS,  and  Gamma)  for  each  particular  dive 
configuration  trial.  The  experimenter  completed  this  exchange  with  a  "Ready  when  you 
are"  statement  The  pilot  then  adjusted  his  stab  trim  and  responded  "Ready"  when  he/she 
was  prepared  for  simulator  release. 

A  set-up  control  interface  program  developed  to  simplify  user-computer  interaction 
allowed  the  experimenter  to  monitor  real-time  characteristics  of  the  simulator  as  it  flew  each 
configuration.  Table  6  presents  an  example  of  the  computer  program  page  that  the 
experimenter  used  to  manipulate  the  terrain  the  simulator  flew  over,  in  addition  to  a  list  of 
the  desired  parameters  for  that  trial. 


Table  6.  An  example  of  the  nilot-in-the-loop  set-up  page. 

RECORD  GCAS  DIVE  RUNS 


1.  TRIAL#  :  24 

2.  SUBJECT  #  :  9 

3.  ROLL  ANGLE  :  9 

4.  GAMMA  :  -15. 

5.  AIRSPEED  :  275. 

6.  TERRAIN  SLOPE  :  14. 

7.  BUFFER  ELEV  :  1000. 

8.  ALTITUDE  :  6000. 


A.  INCREMENT  RUN  :  ON 

B.  QUICK  LOOK  :  ON 

C.  DATA  COL  :  ON 


PRESS  R’  TO  RUN 


Upon  release,  the  pilot  was  required  to  maintain  the  actual  roll  angle  and  gamma  of 
the  simulator  within  2.5  degrees  of  the  desired  parameters  until  warning  initiation.  Failure 
to  be  within  2.5  degrees  of  the  desired  roll  angle  or  gamma  parameters  at  warning  initiation 
or  failure  to  keep  the  maximum  g’s  of  the  aircraft  below  2.5  (see  Design  section)  during 
recovery  would  require  that  trial  to  be  rerun.  No  criteria  were  established  for  the  indicated 
airspeed  parameter,  because  pilots  were  limited  as  to  what  they  could  do  to  maintain 
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airspeed  in  a  steep  (15°)  dive  condition,  especially  under  the  low  (225  knot)  airspeed 
condition.  The  experimenter  monitored  real  time  simulator  performance  characteristics 
using  the  data  page  exhibited  in  Table  3. 

At  warning  initiation,  the  pilot  was  required  to  recover  the  simulator  as  quickly  as 
possible  within  the  operational  limitation  of  the  aircraft.  The  pilot  was  informed  to  continue 
the  recovery  until  a  positive  ground  clearance  rate  of  climb  was  established  on  the  radar 
altimeter.  When  a  positive  radar  altimeter  rate  was  established,  the  experimenter  terminated 
the  trial.  Upon  termination,  the  experimenter  used  a  "quick-look  results"  display  generated 
on  the  experimenter's  console  (Table  7)  to  inform  the  pilot  of  the  actual  dive  angle,  roll 
angle,  and  downward  vertical  velocity  at  warning  initiation,  in  addition  to  the  minimum 
clearance  altitude  reached  during  recovery.  The  pilot  was  then  required  to  make  the 
following  subjective  rating:  "In  your  opinion,  was  the  ground  clearance  provided  by  the 
GCAS  for  your  last  run:  Too  high,  slightly  high,  about  right,  slightly  low,  or  too  low." 


Table,?.  An  example  of  the  pilot-in-the-loop  auick-look  results  display. 
GET  GCAS  EVALUATION 


INITIAL  BA 

5. 

WARN  ALT 

1325.46 

BANK  ANGLE 

4.593 

INITIAL  DA 

-15. 

MIN  CLR 

1000.00 

DIVE  ANGLE 

-9.78 

INITIAL  AS 

225 

ALT  LOST 

325.46 

AIR  SPEED 

248.795 

MAX  G'S 

1.748 

VZ 

-1223.32 

1.  TOO  HIGH 

2.  SLIGHTLY  HIGH 

3.  ABOUT  RIGHT 

4.  SLIGHTLY  LOW 

5.  TOO  LOW 


Enter  evaluation  (RETURN  TO  EXIT): 


The  evaluation  number  was  entered  onto  the  screen  and  then  recorded  as  part  of  the  overall 
trial.  This  information  was  later  used  to  develop  the  pilot  window  of  acceptability 
(discussed  later).  Each  pilot  flew  24  dive  configuration  trials.  The  order  of  trial 
presentation  was  randomized  to  avoid  task  order  effects. 

Part  1  Results 

The  same  variables  collected  for  Phase  2  were  collected  for  the  Part  1  portion  of 
Phase  IB.  These  variables  were  predicted  altitude  loss  due  to  GCASALRT,  GCASDIVE, 
GCASROLL,  and  SAFTHR;  roll,  gamma,  airspeed,  aircraft  g's,  and  terrain  slope  at 
warning  initiation;  and  total  altitude  lost  and  minimum  clearance  during  the  ensuing 
recovery.  One  additional  variable,  the  pilot's  minimum  clearance  rating,  was  also 
collected.  Unlike  Phase  2,  formal  statistical  analyses  were  possible,  since  the  data 
represented  that  of  nine  individual  pilot  subjects,  rather  than  just  one  subject  (robot  pilot). 
Consequently,  the  data  were  not  only  analyzed  for  trend  information,  but  were  also 
subjected  to  multiple  regression  analyses. 

A  mean  minimum  clearance  was  computed  across  all  nine  pilots  for  each  of  the  24 
dive  configuration  trials.  The  resulting  means  were  then  sorted  by  indicated  airspeed  and 
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terrain  slope  and  graphed  as  a  function  of  flight  path  angle  (gamma)  for  all  the  roll  angle 
conditions.  An  inspection  of  the  graphs  (Figures  62-65 )  with  each  other  provided  the 
information  needed  to  determine  die  algorithm’s  ability  to  provide  adequate  minimum 
clearance. 

Airspeed  Effects 

A  comparison  of  Figure  62  with  Figure  63  and  Figure  64  with  Figure  65  indicates 
airspeed  generally  resulted  in  higher  minimum  clearances.  However,  a  repeated  measures 
analysis  of  variance  (see  Table  8)  revealed  airspeed  only  affected  minimum  clearance 
through  its  interaction  with  the  three  other  independent  variables  (gamma,  roll,  and  slope). 
This  could  be  the  result  of  the  pilots  limited  ability  to  control  airspeed  during  the  dive 
configuration  runs.  Consequently,  the  wide  range  of  airspeed  for  the  two  airspeed  levels 
represented  more  of  a  continuous  variable  than  that  of  a  discrete  variable. 

In  each  case,  the  interaction  was  caused  by  a  decrease  in  the  mean  minimum 
clearance  as  airspeed  increased  from  225  to  325.  T-tests  between  the  minimum  clearance 
means  for  the  gamma-airspeed  (t  =  .5487),  roll-airspeed  (t  =  .0455),  and  slope-airspeed 

(t  =  .0842)  interactions  resulted  in  nonsignificant  findings  at  the  .05  a-level.  This  indicates 
the  interactions  were  caused  by  means  not  found  to  be  significantly  different  from  each 
other.  Given  the  mean  minimum  clearances  were  generally  in  the  hypothesized  direction 
and  that  the  continuous  airspeed  variable  was  forced  into  a  dichotomous  variable,  it 
appeared  airspeed  was  adequately  accounted  for  by  the  Cubic  algorithm. 

Table  8.  Repeated  measures  analysis  of  variance  summary  table. 

SflTOCC _ 

Gamma  (G) 

Roll  (R) 

Speed  (Sp) 

Slope  (SI) 

G*R 
G*Sp 
G*S1 
R*Sp 
R*S1 
Sl*Sp 
G*R*Sp 
G*R*S1 
R*Sp*Sl 
G*R*Sp*Sl 


df 

. SS  . . 

F 

D 

2 

7336869 

43.02 

.0001** 

l 

1599432 

47.24 

.0001** 

l 

291347 

2.08 

.1876 

l 

559392 

3.66 

.0923 

2 

105963 

1.06 

.3701 

2 

460731 

6.14 

.0105** 

2 

378010 

5.50 

.0157** 

1 

327185 

6.65 

.0327** 

1 

16093 

.29 

.6077 

1 

339146 

5.49 

.0407** 

2 

0 

.00 

.9999 

2 

41171 

.54 

.5925 

1 

0 

.00 

.9999 

2 

76174 

1.10 

.3577 

♦♦Significant  at  the  .05  a-level. 


Roll  Angle  Effects 

A  review  of  Figures  62-65  and  Table  8  reveals  roll  angle  had  the  hypothesized 
effect  on  minimum  clearance.  The  roll  angle  effect  was  found  to  be  significant  (p=.0001). 
Additionally,  no  interactions  other  than  that  explained  above  occurred.  Accordingly,  we 
concluded  die  effects  of  roll  angle  are  adequately  incorporated  into  the  algorithm. 


GAMMA 


GAMMA 


GCAS  PILOT  RUNS 

IAS=225  SLOPE=14  ELEVATION=1000 


83 


GAMMA 


GAMMA 


Flight  Path  Angle  Effects 


A  review  of  Figures  62-65  reveals  that  gamma  effects  provided  the  expected 
results.  Mean  minimum  clearance  decreased  as  gamma  decreased.  This  effect  was  found 
to  be  statistically  significant  (p=.0001).  Gamma,  however,  was  involved  in  two  different 
two-way  interactions.  The  first  interaction  with  airspeed  was  explained  previously.  The 
interaction  with  slope  will  be  explained  later.  Given  the  general  effect  of  gamma  was  as 
expected  and  later  analyses  indicated  slope  was  the  cause  of  the  interaction,  we  concluded 
gamma  was  adequately  considered  by  the  Cubic  algorithm  for  the  flight  path  angle  range  of 
-5°  to -15°. 


Terrain  Slone  Effects 

Comparisons  of  Figure  62  with  64  and  Figure  63  with  65  provide  trend  information 
indicating  increases  in  slope  resulted  in  increases  in  minimum  clearances.  This  was 
expected.  However,  the  effect  of  slope  on  minimum  clearance  altitude  was  not  significant 
(p=.0923).  Additionally,  slope  was  involved  in  an  unexplainable  interaction  with  gamma. 
Later  analyses  revealed  slope  was  not  adequately  computed  by  the  algorithm. 

Pilot  Window  of  Acceptability 

Pilots'  rating  data  were  divided  into  five  separate  data  sets  and  analyzed 
independently  using  the  linear  regression  analysis  technique.  Minimum  clearance  altitude 
was  considered  the  main  dependent  variable,  while  vertical  velocity  (which  included  two 
critical  factors:  airspeed  and  flight  path  angle)  was  considered  the  main  independent 
variable.  The  five  ratings  corresponded  to  the  statements  found  in  Table  9.  The  regression 
analysis  described  the  relationship  between  minimum  clearance  altitude  as  a  function  of 
vertical  velocity  for  each  of  the  ratings,  by  calculating  the  correlation  coefficient  and  the 
function  of  each  line. 

Table  9.  Ratine  criteria  for  each  minimum  clearance  altitude. 

MINIMUM  CLEARANCE  ALTITUDE 

1.  TOO  HIGH 

2.  SLIGHTLY  HIGH 

3.  ABOUT  RIGHT 

4.  SLIGHTLY  LOW 

5.  TOO  LOW 

Figures  66-70  are  examples  of  the  actual  runs  performed  by  the  various  pilots. 
There  is  an  example  for  each  rating.  For  a  description  of  the  graph  information,  refer  to  the 
Figure  29  discussion  found  in  the  Phase  II  results  section.  One  additional  variable  was 
added  to  the  bottom  of  the  right-hand  column.  This  is  the  rating  (EVAL  NUM)  that  the 
pilot  chose  for  the  given  run.  Additionally,  a  minus  sign  was  added  to  the  roll  value  when 
the  initial  condition  was  a  left  bank  condition.  The  figures  provide  an  idea  of  how  the 
pilous  flew  the  aircraft  A  comparison  of  Figure  66  with  67  supports  our  contention  that  as 
vertical  velocity  increases,  pilots  prefer  increases  in  minimum  clearance  altitudes.  The 
minimum  clearance  of  1483  feet  (Figure  66)  was  rated  "Too  high"  for  a  vertical  velocity  of 
3038  feet  per  minute;  whereas,  the  same  pilot  rated  a  minimum  clearance  of  1357  feet  as 
only  "slightly  high"  for  a  vertical  velocity  of  8886  feet  per  minute. 
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Ffeure  71.  Pilot  window  of  acceptability  with  regression  lines  for  ratings  2. 3.  &  4  and  for  total  data  set. 
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With  Actual  Data  Points 
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PILOT  WINDOW  OF  ACCEPTABILITY 
With  Actual  Data  Points 
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Figure  71  displays  all  of  the  216  data  points  and  each  of  the  regression  lines  for  the 
ratings  of  2,  3,  and  4.  The  figure  also  displays  the  overall  line  of  prediction  of  the  Cubic 
algorithm.  To  provide  a  graph  of  adequate  size,  four  data  points  were  not  shown.  Three 
of  these  data  points  represented  minimum  clearances  greater  than  1500  feet  and  the 
remaining  data  point  represented  a  very  high  dive  rate  (1 1,000  ft/min)  for  the  KC-135 
aircraft  However,  these  data  points  were  included  in  the  appropriate  regression  analyses. 

The  upper  limit  of  the  window  of  acceptability  was  determined  by  regressing  the 
minimum  clearance  altitudes  onto  the  vertical  velocity  for  all  trials  given  a  rating  of  2 
(slightly  high).  The  lower  limit  of  the  window  was  calculated  in  a  similar  manner  for  a 
rating  of  4  (slightly  low).  The  regression  line  for  the  rating  of  3  indicates  what  the  pilots 
felt  was  the  appropriate  regression  line  of  prediction  (Y=.0623X  +  293).  The  overall  line 
of  prediction  is  the  regression  line  for  every  trial  (N=216)  and  is  represented  by  the 
equation  Y=.087X  +  146.  It  is  the  Cubic  algorithm  line  of  prediction.  The  resulting 
regression  line  for  the  upper  limit  was  Y=.0919X  +  280;  the  lower  limit, 

Y=.0353X +  219.  All  the  regression  lines  were  analyzed  for  higher  order  trends.  Linear 
models  proved  to  be  the  most  appropriate  models  for  all  of  the  data  sets. 

The  "window  of  acceptability"  is  the  area  located  between  the  slightly  high  and  the 
slightly  low  regression  lines  and  represents  the  region  that  pilots  feel  relatively  comfortable 
with  the  minimum  clearance  provided  by  the  algorithm  for  the  given  vertical  velocity. 
Minimum  clearance  altitudes  greater  than  the  upper  limit  would  be  considered  too  high 
(false  alarms).  Minimum  clearances  below  the  lower  limit  would  be  considered  too  low 
(unsafe-too  close  for  comfort).  As  Figure  71  shows,  the  variability  of  the  minimum 
clearances  provided  by  the  algorithm  was  excessive.  Thirty-one  percent  of  all  of  the 
minimum  clearances  were  false  alarms.  Thirty-one  percent  of  the  minimum  clearances 
were  "too  close  for  comfort."  Only  thirty-eight  percent  of  the  actual  minimum  clearances 
provided  were  within  the  window  of  acceptability. 

To  determine  the  source  of  this  variability,  the  pilot  window  of  acceptability  was 
analyzed  as  a  function  of  slope.  Figures  72  and  73  are  examples  of  the  pilot  window  of 
acceptability  with  only  the  data  points  for  the  given  slope  condition  plotted.  A  decrease  in 
the  variability  is  observed  for  a  slope  of  seven  (Figure  72).  Fifty-two  percent  of  the  data 
fell  within  the  window  of  acceptability.  Only  20%  of  the  data  points  were  false  alarms, 
while  28%  were  "too  close  for  comfort"  This  is  an  improvement  over  the  entire  data  set 
On  the  other  hand,  the  variability  of  the  data  set  for  a  slope  of  14  (Figure  73)  actually 
increased  in  comparison  to  the  the  total  data  set.  Only  26%  of  these  data  points  fell  within 
the  pilot  window  of  acceptability.  The  false  alarm  rate  was  42%  and  the  "too  close  for 
comfort"  rate  was  32%. 


Part  1  Discussion 

The  statistical  analyses  and  trend  analyses  for  Part  1  indicated  the  inability  of  the 
Cubic  algorithm  to  provide  consistent,  adequate  minimum  clearances  for  the  KC-135.  The 
algorithm  was  only  able  to  provide  a  pilot-acceptable  minimum  clearance  38%  of  the  time. 
The  algorithm  generated  minimum  clearances  considered  too  close  for  comfort  31%  of  the 
time.  Four  of  these  resulted  in  ground  collisions.  The  false  alarm  rate  was  also  31%. 

The  follow-on  analysis  for  slope  effects  revealed  the  algorithm’s  inability  to 
accurately  account  for  the  effects  of  slope  caused  increased  variability  in  the  minimum 
clearance  altitudes.  Given  the  above  results,  it  is  highly  questionable  whether  the  pilot 
would  place  any  confidence  in  this  algorithm.  The  Cubic  algorithm  needs  to  improve  its 
ability  to  accurately  account  for  the  effects  of  slope  given  various  aircraft  configurations. 
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Part  2  -  Approach  and  Landing 


Procedure 

Upon  completion  of  the  Part  1  dive  configuration  trials,  the  pilot  was  given  a  short 
rest  period.  The  pilot  was  then  given  an  indepth  briefing  describing  the  ILS  approach  to  be 
performed.  Prior  to  the  actual  recording  of  any  data  the  pilot  was  allowed  to  practice  as 
many  ELS  approaches  as  desired.  The  practice  period  was  terminated  when  the  pilot  felt 
comfortable  in  his/her  ability  to  fly  the  ILS  approach  proficiendy. 

The  pilot  was  required  to  fly  an  ILS  approach  to  Runway  24R  at  Los  Angeles 
(Figure  74).  All  radios  and  ILS  frequencies  were  preset  prior  to  the  start  of  each  run.  The 
KC-135  simulator  was  positioned  at  a  starting  point  14  NM  Northeast  of  the  runway  at  an 
altitude  of  2000  feet.  This  required  the  pilot  to  intercept  both  the  glideslope  and  localizer 
course.  The  Night  Visual  System  described  earlier  was  active;  however,  simulated  weather 
prevented  the  pilot  from  identifying  the  runway  lights  until  a  descent  below  300  feet  was 
made.  This  forced  the  pilot  to  use  the  instruments  displaying  ILS  information  and 
prevented  him  from  using  outside  visual  cues  during  die  approach.  The  pilots  flew  a  total 
of  eight  approaches  for  the  various  aircraft  configurations  of  flaps  (up  or  down),  gear 
(up  or  down),  and  forced  glideslope  deviations  (yes  or  no). 

Data  Collection 

Data  collection  for  the  ILS  approach  was  started  at  glideslope  intercept  or  at  1000 
feet  (AGL),  whichever  occurred  first.  It  continued  on  a  30  Hz  cycle  until  100  feet  (AGL). 
The  data  set  used  for  analysis  was  based  on  individual  data  points  recorded  every  fifteen 
seconds,  and  for  every  crash  and  warning.  The  recorded  information  formed  the  general 
categories:  algorithm's  calculations,  and  actual  aircraft  conditions.  The  algorithm 
calculation  variables  were:  warning  (yes  or  no),  time  of  data  recording,  reaction  time  (ft), 
roll  (ft),  gamma  (ft),  safety  buffer  (ft),  total  altitude  loss  (ft),  and  pilot  reaction  time  (sec). 
The  actual  aircraft  conditions  were:  roll  (degrees),  gamma  (degrees),  altitude  (ft-AGL), 
airspeed  (knots),  vertical  velocity  (ft/min),  terrain  slope  (degrees),  minimum  clearances 
(ft-AGL),  Maximum  G’s,  total  altitude  lost  (ft),  pilot  reaction  time  (sec),  and  G’s.  The 
period  30  seconds  before  until  60  seconds  after  the  warning  defined  the  data  collection 
window. 


Part  2  Results 

The  resulting  data  set  consisted  of  309  data  points.  These  data  were  individually 
inspected  for  the  actual  airspeed,  flap,  gear,  glideslope  deviation,  and  altitude  conditions 
that  existed.  Based  on  the  actual  conditions  existing,  it  was  determined  whether  a  warning 
should  have  been  generated,  and  if  so,  what  type  of  warning  should  have  been  generated 
(see  "Cubic  GCAS"  section).  A  comparison  of  what  should  have  occurred  with  what 
actually  happened  was  then  made.  This  allowed  the  experimenters  to  determine  the  success 
rate  of  the  algorithm  in  accurately  providing  a  warning  for  the  approach  and  landing  phase 
of  flight 

Out  of  309  possible  data  points,  298  correct  identifications  were  made.  A  correct 
identification  was  made  when  no  warning  was  provided  and  the  actual  conditions  at  the 
time  dictated  that  no  warning  be  provided  or  a  warning  was  provided  when  a  warning 
should  have  been  provided.  This  represented  a  success  rate  of  96.4%. 
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The  failure  of  the  algorithm  to  identify  1 1  data  points  was  broken  into  two 
categories.  The  first  category  was  called  "false  alarms."  This  category  represents  the 
times  when  the  algorithm  provided  a  warning  and  no  warning  should  have  been  given.This 
occurred  seven  times  for  a  false  alarm  rate  of  2.3%.  The  other  category,  "misses," 
represents  the  occasions  when  the  algorithm  failed  to  generate  a  warning  when  a  warning 
should  have  been  applied.  This  occurred  4  of  309  times  for  a  miss  rate  of  1.3%.  The  total 
failure  rate  of  the  approach  and  landing  sub- algorithm  was  3.6%. 

A  breakdown  of  the  misses  and  false  alarms  revealed  all  the  misses  were  a  result  of 
the  algorithm's  inability  to  accurately  provide  warnings  for  glideslope  deviation.  Three 
glideslope  warnings  were  inhibited  due  to  other  warnings.  No  priority  of  warnings  existed 
in  the  Cubic  GCAS  algorithm.  Consequendy,  the  algorithm  simply  prioritized  the 
warnings  through  alphabetization.  Only  one  miss  occurred  from  the  algorithm's  failure  to 
provide  a  glideslope  warning  for  a  glideslope  deviation  to  the  low  side.  Of  the  seven  false 
alarms,  two  were  the  result  of  a  glideslope  warning  being  given  below  a  250-foot  altitude. 
As  explained  earlier,  the  minimum  altitude  for  which  a  glideslope  warning  is  to  be  given  by 
the  Cubic  GCAS  algorithm  is  250  feet  (AGL).  The  remaining  five  false  alarms  resulted 
from  a  "pull-up"  warning  being  generated  whenever  the  pilot  bounced  the  aircraft  on 
landing  and  then  attempted  to  force  the  aircraft  back  down  for  the  final  full  stop.  Although 
the  conditions  would  dictate  a  "pull-up"  be  generated,  a  warning  at  such  a  critical  time  was 
considered  to  be  a  nuisance,  since  impact  (touchdown)  with  the  ground  is  the  intent  of  the 
landing.  The  incorporation  of  an  inhibit  would  be  appropriate. 

Part  2  Discussion 

Phase  III  evaluated  the  Cubic  algorithm's  ability  to  accurately  provide  safe 
minimum  clearances  to  a  real  pilot  in  a  consistent  manner.  By  including  the  human  pilot 
into  the  evaluation,  we  were  able  to  evaluate  the  current  algorithm  with  the  human  factor 
concern  considered  in  the  design.  This  allowed  the  formulation  of  the  pilot  window  of 
acceptability.  This  window  should  be  useful  to  the  algorithm's  designers,  given  it 
graphically  describes  the  trend  of  the  data,  making  areas  needing  additional  development 
more  obvious.  It  also  provides  the  SPO  engineers  with  a  preliminary  tool  for  evaluating 
the  false  alarm  criteria  set  forth  in  the  statement  of  work.  The  window  of  acceptability 
should  be  developed  further  by  increasing  the  number  of  pilots  participating  in  the 
evaluation.  This  would  provide  a  window  of  acceptability  more  representative  of  the  pilot 
crew  force  and  of  the  warnings'  relationships  with  minimum  recovery  altitudes  and  vertical 
velocity. 

Our  evaluation  determined  the  Cubic  algorithm  provides  inconsistent  minimum 
clearances  resulting  in  a  high  degree  of  variability.  Further  analysis  revealed  the  algorithm 
failed  to  adequately  account  for  the  effects  of  slope  during  high  terrain  slope  conditions. 

An  analysis  of  the  GCASLAND  and  TCHDWN  subroutines  revealed  these  subroutines  had 
a  prediction  accuracy  rate  of  96.4%.  However,  the  pilots  suggested  a  glideslope  warning 
for  a  deviation  to  the  high  side  of  glideslope  would  also  prove  very  beneficial  and  should 
be  recommended.  An  inhibit  of  the  "pull-up"  message  during  a  bounce  on  landing  should 
also  be  incorporated.  Additionally,  some  priority  of  message  warning  should  be 
established  within  the  Cubic  algorithm. 


97 


PHASE  IV 


The  objective  of  the  fourth  phase  of  the  evaluation  was  to  introduce  the  pilot  to  a 
full  mission  using  an  actual  terrain  database.  By  doing  so,  the  pilot  was  able  to  fly  the 
simulator  under  normal  flight  conditions  over  terrain  and  obtain  real  time  radar  altimeter 
information.  This  scenario  allowed  the  GCAS  algorithm  to  be  evaluated  under  normal 
operational  flight  profiles,  and  also  helped  reduce  possible  pilot  response  biases  that  might 
have  occurred  in  Phase  III.  Response  bias  would  be  reduced  because  Phase  IV  pilots 
would  be  uncertain  of  an  upcoming  GCAS  warning;  whereas.  Phase  III  pilots  were 
guaranteed  to  hear  a  warning  on  every  dive  configuration  trial.  The  procedure  and  results 
of  Phase  IV  follow. 


Method 


Subjects 

Eight  of  the  nine  pilots  from  Phase  III  of  our  evaluation  were  again  used  during 
Phase  IV.  The  one  pilot  not  involved  in  Phase  IV  was  used  as  a  checkout  pilot  for  the  full 
mission  scenario. 

Apparatus 

Facility.  Refer  to  the  Phase  II  Apparatus  section  for  a  description. 

Computer  Complex.  Refer  to  the  Phase  II  Apparatus  section  for  a  description. 

Simulator.  Refer  to  the  Phase  II  and  Phase  III  Apparatus  sections  for  a 
description  of  the  KC-135  simulator.  In  addition,  a  computer  program  read  a  Defense 
Mapping  Agency  (DMA)  terrain  database  into  memory  of  a  Gould  Sel  87  computer.  The 
elevation  of  the  terrain  was  then  computed  based  on  an  extrapolation  of  the  simulator’s 
position  in  relationship  to  the  database.  The  subtraction  of  the  terrain  elevation  from  the 
aircraft's  barometric  altitude  (computer  based)  provided  the  above  ground  information  fed 
back  to  the  aircraft's  radar  altimeter  indicator. 

Experimenters  Console.  Refer  to  the  Phase  III  Apparatus  section  for  a 
description. 

Voice  Message  Unit  Mechanization.  Refer  to  the  Phase  III  Apparatus 
section  for  a  description. 

Audio  Systems.  Refer  to  the  Phase  III  Apparatus  section  for  a  description. 

Visual  Warning  Signal.  Refer  to  the  Phase  III  Apparatus  section  for  a 
description. 

Eracedure 

This  portion  of  the  evaluation  was  performed  on  the  same  day  as  Phase  HI. 
However,  the  pilots  flew  the  Phase  IV  full  mission  during  the  afternoon  after  a  minimum  of 
a  one  hour  lunch  break.  Upon  return,  the  pilot  was  given  a  standardized  briefing 
concerning  the  requirements  of  the  mission.  No  additional  simulator  training  was  given  the 
pilot  since  ample  flight  time  had  resulted  from  the  Phase  III  portion  of  the  evaluation. 

None  of  the  pilots  considered  this  a  problem. 
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The  mission  required  the  pilot  to  takeoff  on  Runway  25R  at  Los  Angeles  and  begin 
a  climb  to  12,000.  At  2,000  feet,  the  pilot  was  required  to  turn  right  and  fly  direct  to  Santa 
Catalina  tacan.  From  that  point  on,  the  pilot  was  briefed  to  fly  the  planned  route  and 
altitudes,  unless  told  by  the  navigator,  LA  center,  or  the  experimenter  (all  roles  were  played 
by  the  experimenter)  to  deviate  from  that  planned  (See  Figure  75).  Multiple  interactions 
and  interventions  with  the  navigator  and  LA  center  occurred  throughout  the  flight  to 
simulate  realistic  flight  communications.  The  interactions  were  both  planned  and  random  in 
nature.  Pilots  were  informed  an  intervention  by  center  or  the  navigator  did  not  necessarily 
indicate  a  warning  was  forthcoming.  On  the  contrary,  less  than  twenty  percent  of  all  the 
ground  impacts  or  warnings  occurred  outside  the  two  minute  period  following  an 
intervention. 

The  pilot  was  normally  required  to  fly  the  aircraft  at  altitudes  under  1000  feet 
(AGL)  in  an  effort  to  place  the  aircraft  in  an  environment  where  GCAS  warnings  would 
occur.  Although  this  environment  is  not  typical  for  a  tanker,  none  of  the  pilots  felt  that  it 
represented  a  major  difficulty,  since  they  were  only  required  to  fly  the  aircraft  under  normal 
flight  profiles  (i.e.,  straight  and  level,  standard  climbs  and  descents).  The  pilot  was 
allowed  full  use  of  his  instruments  with  the  exception  that  the  radio  altimeter  bug  switch 
had  to  be  set  to  less  than  one  hundred  feet,  so  the  light  would  not  allow  the  pilot  to 
anticipate  a  possible  GCAS  warning.  During  flight,  the  experimenter  could  continually 
track  aircraft  position  and  configuration  through  the  experimenter’s  console  (Table  10). 


Table  10.  An  example  of  the  mission  data  display  page  for  Phase  IV. 


FLY  DATA  DISPLAY  PAGE 


GAMMA 

.0 

PITCH 

.0 

ALT 

135 

ROLL 

.0 

AOA 

.0 

RALT 

9 

IAS 

3 

GS 

1.00 

VVI 

-2 

WARN:  (NO  WARNING) 

KC135  STATUS  LANDING  CONDITIONS  DATA  COLLECTION 


(FLYING) 

TCN  CH  83.0X 
BRG  48.0 


GEAR 

0% 

RUN  : 

1 

FLAPS 

5% 

DATA  : 

(INACTIVE) 

GS  DEV 

-3.0 

LOCDEV 

.0 

DME 

.0 

Press  ATTENTION  to  EXIT  DATA 


In  the  event  of  a  crash,  the  pilot  would  continue  to  fly  the  simulator  as  though  the 
impact  with  the  ground  had  not  occurred.  The  pilot  and  experimenter  were  made  aware  of 
a  crash  by  the  zeroing  of  the  radio  altimeter  with  a  simultaneous  clicking  sound  as  the 
aircraft  impacted  the  terrain.  Under  extremely  hard  impacts  or  well  below  ground  impacts, 
the  simulator  was  required  to  be  reset.  The  experimenter  would  reset  the  simulator  at  the 
point  of  impact,  but  at  an  altitude  high  enough  to  avoid  an  immediate  crash  with  the 
ground.  Hie  pilot  was  then  briefed  on  the  aircraft  position  and  configuration  and  asked  to 
state  when  he  was  ready  for  release.  When  the  pilot  was  ready,  the  simulation  would 
continue  and  the  pilot  would  again  fly  the  simulator  as  briefed.  The  planned  mission  was 
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scheduled  for  two  and  one  half  hours.  Due  to  numerous  crashes  for  each  pilot,  no  pilot 
successfully  completed  the  entire  mission.  When  the  simulation  portion  of  Phase  IV  was 
terminated,  the  pilot  was  given  two  questionnaires  to  gather  their  subjective  opinions 
concerning  the  implementation  of  the  GCAS,  the  reliability  of  this  GCAS  algorithm,  and 
the  quality  of  the  simulation  provided.  These  questionnaires  can  be  found  in  Appendix  B. 
The  pilot  was  then  debriefed  and  thanked  for  his/her  participation  in  Phases  III  and  IV  of 
our  evaluation. 

Data  Collection 

Data  were  collected  on  a  15  Hz  cycle  for  the  duration  of  the  mission.  Two 
categories  of  variables  were  recorded:  Algorithm  calculations  and  actual  parameters.  These 
are  explained  in  more  detail  in  the  results  section.  In  order  to  make  the  data  set  manageable 
and  to  adequately  evaluate  the  algorithm's  predictive  ability,  only  data  points  where  an 
actual  GCAS  warning  was  given  or  when  a  crash  occurred  were  placed  into  the  database 
for  final  analysis.  This  resulted  in  a  database  of  82  data  points.  The  data  collection 
window  for  a  given  data  point  was  the  period  30  seconds  before  the  event  to  sixty  seconds 
after  the  event.  Minimum  clearances  and  maximum  g's  were  based  on  the  window. 

Results 

The  data  were  sorted  by  minimum  clearances,  gamma,  and  terrain  slope.  A 
frequency  analysis  revealed  the  82  data  points  consisted  of  78  warnings,  2  crashes  with  no 
warning,  and  2  false  alarm  warnings.  A  review  of  Table  1 1  reveals  that  39  of  the  78 
warnings  resulted  in  ground  impacts.  With  the  two  crashes  that  received  no  warning,  the 
final  total  is  41  of  82  (50%)  possible  data  points  resulted  in  aircraft  crashes. 


Table  11. _ Phase  IV^Full  mission  data  point  frequency  breakdown. 

82  -  Total  data  points 

2  -  Crashes  with  no  warning  generated 
2  -  False  Alarms 
78  -  Warning  generated 
20  -  Valid  warnings 
39  -  Crashes 

10  -  Below  50  feet  (AGL) 

9  -  50  <  MinclrclOO 


The  data  were  further  analyzed  by  breaking  down  the  crashes  by  gamma.  An 
analysis  of  Table  12  shows  the  majority  of  the  crashes  were  for  shallow  dive  angles  or  for 
climbing  flight  path  angles.  Our  previous  phases  of  the  evaluation  had  focused  on  dive 

angles  equal  to  or  less  than  -5°  nose  down.  Table  12  indicates  the  Cubic  algorithm  is 
having  difficulty  providing  safe  minimum  clearances  for  flight  path  angles  between  -5° 
nose  down  to  10°  nose  up.  For  example,  only  one  crash  occurred  for  flight  path  angles 
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less  than  -5°  (dives),  but  13  crashes  occurred  for  flight  path  angles  between  -1°  and  0° 
(shallow  dive-near  straight  and  level  flight).  A  review  of  the  flight  path  angles  (climbs) 

greater  than  5°  (Table  12:  5<y  Breakdown)  further  indicates  the  crashes  were  the  result  of 
the  terrain  slope  climbing  faster  than  the  aircraft.  The  algorithm  was  unable  to  adequately 
account  for  this  condition. 

Table  12.  Phase  IV  -  Full  mission  crash  frequency  breakdown. 

39  Crashes 

1  -  Gamma  (y)  <  -5  8  —  y  <  1 

5  - y < -2  0-y<5 

7-yc-l  4-y<10 

13-ycO  1  —  10  <  y 


5  <  y 

Breakdown 

y  =  +5.16 

Slope  =  11.71 

y  =  +6.50 

Slope  =  9.95 

y  =  +6.62 

Slope  =  12.51 

y  =  +8.36 

Slope  =  +9.95 

y  =  +11.09 

Slope  =  15.89 

The  aircraft 's  actual  flight  path  in  relationship  to  the  terrain  database  was  also 
plotted  for  each  of  the  data  points.  Figures  76-80  are  some  examples.  They  display  the 
subject  number  at  the  top  of  the  page.  The  first  five  lines  along  the  side  of  the  graph  were 
the  algorithm's  predicted  altitude  loss  for  each  of  the  subroutines,  GCASALRT, 
GCASROLL,  and  GCASDIVE,  in  addition  to  the  algorithm's  calculated  safety  buffer. 
These  are  then  summed  to  obtain  the  total  predicted  altitude  loss,  ALT  LOSS.  The  first 
reaction  time  listed  was  that  used  by  the  algorithm.  The  roll,  gamma,  altitude,  airspeed, 
and  vertical  velocity  (VZ),  found  in  the  second  group  of  variables,  were  the  actual 
conditions  of  the  aircraft  simulator  at  warning  initiation.  Maximum  G's  (MAX  Gs)  and 
actual  total  altitude  loss  (ALT  LOSS)  were  the  maximum  values  obtained  during  aircraft 
recovery.  Minimum  clearance  (MIN  ALT)  represents  the  minimum  distance  between  the 
aircraft  and  the  terrain  that  occurred  during  the  aircraft's  recovery.  The  roll  line  and  G's 
line,  at  the  upper  portion  of  the  graph,  represent  the  realtime  roll  and  g-load  values  of  the 
aircraft  during  its  flight.  The  x-axis  is  the  total  running  time  since  the  mission  began.  The 
y-axis  is  the  altitude  of  the  aircraft  in  feet  (MSL).  The  flight  of  the  aircraft  is  represented  by 
the  curved  upper  line  beginning  at  the  higher  region  of  the  y-axis.  The  ground  level  is 
represented  by  the  jagged  line  that  begins  at  the  lower  region  of  the  y-axis. 

Figure  76  provides  an  excellent  example  of  a  false  alarm.  The  simulator  is  flying  in 
essentially  a  straight  and  level  configuration,  while  the  terrain  is  rapidly  increasing.  The 
warning  is  received  at  the  moment  the  terrain  levels  off.  The  result  was  a  false  alarm. 
Figure  77  provides  an  example  of  the  aircraft  flying  essentially  a  straight  and  level  flight 
profile  with  rising  terrain.  The  warning  was  provided  early  enough  to  allow  for  a 
minimum  clearance  altitude  of  267  feet.  This  figure  also  provides  an  excellent 
representation  of  one  of  several  mountain  ridges  the  pilots  were  required  to  fly  over  during 
the  mission. 
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Figure  76.  Example  of  a  "false  alarm"  data  point. 


Figure  78  provides  an  example  of  a  shallow  descent  profile  into  gently  rising 
terrain.  No  warning  was  provided.  The  recovery  was  initiated  because  the  pilot  noticed 
the  drop  in  altitude  on  the  radio  altimeter  and  responded  accordingly.  Figure  79  shows  the 
algorithm's  failure  to  provide  the  pilot  with  a  warning  early  enough  to  affect  recovery. 
Figure  80  is  an  example  where  the  pilot  began  a  climb  as  directed  by  the  navigator.  At  the 
start  of  the  climb,  no  warning  had  been  sounded.  A  warning  was  finally  generated  during 
the  climb,  but  it  was  just  moments  before  impact.  This  provides  another  example  of  the 
algorithm's  inability  to  account  for  flight  path  angles  of  shallow  descent  or  climb. 

Given  the  algorithm's  inability  to  account  for  flight  path  angles  greater  than  -5°,  one 
final  analysis  was  run  in  an  attempt  to  isolate  a  possible  source  of  the  problem.  In  a  similar 
fashion  to  the  robot  runs  of  Phase  II,  several  GCAS  warning  trials  were  performed  using 
the  robot  model.  Unlike  Phase  II,  the  roll,  gamma,  and  terrain  elevation  variables  were 

kept  constant  at  values  of  0°,  2°,  and  1000  feet,  respectively.  The  independent  variables 
were  airspeed  (225,  275,  &  325  knots)  and  slope  (3.5, 7, 10.5,  &  14  degrees)  with  the 
dependent  variables  of  interest  being  minimum  clearance  altitude  and  warning  initiation 
altitude. 

Figures  8 1  and  82  provide  us  with  pictorial  representations  of  the  results.  If  the 
algorithm  was  performing  calculations  based  on  terrain  slope,  the  warning  altitudes  should 
have  increased  as  the  slope  of  the  terrain  increased.  A  glance  at  Figure  81  shows  just  that. 
The  warning  altitude  did  increase  as  the  slope  of  the  terrain  increased,  across  all  airspeeds. 
Since  the  algorithm  is  considering  slope  effects,  then  it  may  be  the  algorithm  failed  to 
calculate  the  effects  of  slope  accurately.  Figure  82  indicates  this  is  the  underlying  problem. 
If  the  algorithm  were  accounting  for  the  effect  of  slope  accurately,  we  would  expect 
minimum  clearance  to  increase  as  slope  increased  (explained  earlier  in  Phase  II).  This  is 
not  what  Figure  82  indicates.  Rather  than  an  increase  in  the  minimum  clearances,  there 
was  a  decrease  in  the  minimum  clearances.  Additionally,  all  of  the  minimum  clearances  for 
slopes  greater  than  3.5°  resulted  in  crashes  (Figure  82). 

Discussion 

The  results  of  Phase  IV  demonstrated  the  algorithm's  inability  to  accurately  account 
for  the  effects  of  slope.  This  inability  was  due  to  the  algorithm  not  providing  adequate 
increases  in  warning  altitude  as  a  result  of  slope  effects.  This  problem  was  farther 
aggravated  under  flight  path  angle  conditions  greater  than  -5°.  Additional  effort  must  be 
directed  at  correcting  this  deficiency.  Additionally,  two  false  alarms  occurred  which 
resulted  in  minimum  clearances  greater  than  1 800  feet  Further  refinement  of  the  algorithm 
should  consider  the  possibility  that  an  over  correction  for  the  effects  of  slope  may  cause  an 
increase  in  false  alarms.  This  would  also  be  an  undesirable  situation  as  pilot  confidence  in 
the  algorithm  would  be  greatly  undermined.  Future  efforts  should  consider  the  possibility 
of  such  a  trade-off. 
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Figure  79.  Example  of  a  "straight  and  level  flight-late  warning"  data  point. 
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GCAS  ROBOT  RUNS 
Gamma=2  Roll=0  Elevation= 


PILOT  QUESTIONNAIRES 


At  the  completion  of  Phase  IV,  the  subjects  were  required  to  answer  the  two 
questionnaires  found  in  Appendixes  B  &  C.  The  questionnaire  titled  "Ground  Collision 
Avoidance  System  Questionnaire  For  Cargo/Tanker  Type  Aircraft"  (Appendix  B)  was 
originally  mailed  out  to  operational  KC-135  bases  to  gather  aircrew  subjective  data 
concerning  the  possible  implementation  of  a  GCAS  system  into  the  KC-135  aircraft.  Some 
of  the  results  of  this  questionnaire  (Rueb  &  Hassoun,  1990)  provided  valuable  information 
for  designing  the  simulator  warning  signals  used  for  this  study  within  current  military 
requirements  (Mil  Std  1800,  Mil  Std  41  ID).  The  second  questionnaire  was  used  to 
evaluate  the  algorithm  and  the  CSEF  simulator. 

The  "KC-135  Ground  Collision  Avoidance  Questionnaire"  results  for  this  study 
(N=9,  hereafter  referred  to  as  Study  B)  paralleled  that  of  the  original  study 
(N=82,  hereafter,  referred  to  as  Study  A)  with  the  following  exceptions.  First,  the  pilots 
(8  of  9)  in  Study  B  definitely  felt  a  voice  warning  was  the  preferred  aural  mode  of 
warning.  There  was  a  fairly  even  split  between  voice  and  tone  preference  for  Study  A. 
Study  B  respondents  also  felt  the  GCAS  should  not  be  able  to  be  shutoff.  Instead,  they  felt 
a  reset  capability  would  be  advantageous.  Additionally,  the  Study  B  pilots  felt  the 
maximum  altitude  coverage  of  the  GCAS  should  be  limited  to  the  maximum  coverage  of  the 
radar  altimeter,  whereas ,  Study  A  indicated  a  maximum  coverage  of  5000  feet  would  be 
appropriate.  Given  the  pilots  in  Study  B  had  actual  experience  with  a  GCAS,  we  placed 
more  credence  in  the  idea  that  the  maximum  altitude  of  the  GCAS  be  tied  directly  to  that  of 
the  radio  altimeter. 

Another  difference  between  the  Study  A  population  and  Study  B's  subject 
population  is  the  response  given  to  the  nomenclature  that  should  be  printed  on  the  warning 
light.  Study  A  respondents  preferred  "Altitude;"  Study  B  pilots  preferred  "Pull-up."  Since 
they  preferred  the  voice  warning,  "Pull-up,"  used  in  the  study,  they  felt  the  light 
nomenclature  should  also  reflect  "Pull-up."  This  finding  supports  the  findings  of 
Hassoun,  Kinzig,  &  Bamaba,  (1988). 

With  the  exceptions  noted  above,  the  Study  B  responses  validated  the  subjective 
responses  provided  in  Study  A.  Recall  Study  B  subjects  had  actually  flown  a 
GCAS-equipped  simulator.  Study  A  respondents  had  not  had  any  experience  with  a 
GCAS  system.  This  is  important  because  it  substantiates  the  utility  of  questionnaire  data. 

The  results  of  the  untitled  second  questionnaire  (Appendix  C)  supported  the 
findings  of  this  evaluation.  Specifically,  the  pilots  felt  the  overall  ground  clearance  of  the 
Cubic  GCAS  algorithm  was  slightly  low  (5  of  9).  They  also  felt  the  Cubic  algorithm 
applied  its  warnings  inconsistently  (7  of  9),  and  consequently,  were  not  confident  (6  of  9) 
in  the  algorithm.  This  is  a  direct  result  of  the  high  variability  identified  during  Phases  m 
and  IV  of  our  evaluation.  The  subjects  did,  however,  believe  the  use  of  the  Cubic  GCAS 
algorithm  would  still  be  beneficial  for  the  KC-135  aircraft.  This  was  based  primarily  on 
their  ideas  that  any  system  that  prevented  even  one  accident  is  beneficial. 

The  whole  premise  underlying  this  study  is  based  on  this  simulator’s  fidelity  and 
realism.  The  degree  that  the  performance  of  the  simulator  mirrors  that  of  the  aircraft 
represents  the  degree  of  confidence  that  we  can  place  in  our  findings.  Two  subjects  found 
the  simulator  performance  to  be  excellent,  five  found  it  to  be  good.  Only  one  felt  the 
simulator  was  fair  in  performance,  but  did  not  state  why.  Based  on  the  results  to  this 
question,  we  can  place  a  high  degree  of  confidence  in  this  study's  findings. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Since  the  problems  of  Phase  I  and  Phase  II  were  corrected  prior  to  the  next  phase  of 
the  evaluation,  this  section  will  focus  only  on  our  findings  and  recommendations  for 
Phases  III  and  IV.  After  an  extensive  and  in-depth  analysis,  several  problem  areas  were 
identified.  Specifically,  the  Cubic  GCAS  algorithm: 

a.  Provided  too  much  variability  in  the  minimum  clearance  altitudes  under  high 
terrain  slope  conditions. 

b.  Failed  to  account  for  the  effects  of  slope  under  low  dive  angles  and  under  climb 
conditions. 

c.  Failed  to  provide  for  message  priority. 

d.  Failed  to  provide  a  warning  to  the  high  side  of  glideslope. 

e.  Provided  a  nuisance  "pull-up"  warning  for  bounces  that  occurred  during 
landing. 

f.  Failed  to  provide  the  pilot  with  enough  time  to  recover  the  aircraft  for  landing  on 
that  approach,  based  on  the  current  flap  and  gear  warning  being  given  at  500  feet 
AGL. 

Based  on  our  findings,  we  recommend  the  current  version  of  the  Cubic  GCAS 
algorithm  not  be  used.  The  following  corrections  to  the  Cubic  algorithm  should  be  made: 

a.  The  possible  inclusion  of  the  effects  of  Cg  into  the  algorithm  should  be 

considered. 

b.  The  algorithm  should  be  adjusted  to  better  account  for  the  effects  of  slope. 

c.  The  algorithm  should  cover  a  flight  path  angle  range  of  20  degrees  nose  down  to 
15  degrees  nose  up. 

d.  The  algorithm  should  provide  a  warning  for  glideslope  deviations  to  the  high 
side  of  glideslope. 

e.  The  algorithm  should  provide  a  minimum  ground  clearance  of  150  feet  (AGL) 
and  a  maximum  ground  clearance  of  1000  feet  (AGL).  A  general  rule  of  thumb 
is  approximately  10%  of  the  downward  vertical  velocity  of  the  aircraft  at 
warning  initiation. 

f.  Measures  should  be  included  to  inhibit  the  pull-up  message  for  possible  bounces 
on  the  runway  during  landing. 

g.  Finally,  if  the  approach  and  landing  algorithm  is  intended  to  provide  the  pilot 
adequate  time  to  effectively  land  the  aircraft,  then  the  gear  and  flap  warnings 
should  begin  at  1000  feet  (AGL),  instead  of  the  500  feet  currently  used  by  the 
algorithm. 
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SUMMARY 


The  Cubic  GCAS  algorithm  was  evaluated  in  a  four  phase  simulation  effort. 

Phase  I  attempted  to  verify  and  validate  the  sub-algorithms.  In  Phase  II,  a  robot  pilot 
model  was  used  to  test  the  algorithm  in  the  KC- 1 35  simulator.  The  robot  pilot  model 
performed  various  dive  recovery  maneuvers.  A  subset  of  which  were  used  in  Phase  III. 
Phase  III  and  Phase  IV  involved  man-in-the-loop  simulation.  At  the  conclusion  of  Phases 
I,  n,  and  IV,  recommendations  were  made  to  the  SPO  and  the  Cubic  Corporation  regarding 
possible  refinements  of  the  algorithm. 

Hassoun,  Ward,  Bamaba,  &  McCarthy  (1989)  posed  the  question  in  reference  to 
the  pilot  window  of  acceptability,  "Can  we  generalize  from  the  present  study  results,  or 
should  a  new  set  of  data  be  collected  on  each  individual  aircraft  ?"  (p.  1 50)  Based  on  the 
results  of  our  study,  initial  indications  suggest  the  pilot  window  of  acceptability  should  be 
generated  based  on  individual  aircraft.  A  comparison  of  the  FB-1 1 1  and  the  KC-135 
windows  of  acceptability  (Figures  83  &  84)  reveals  significant  differences  in  minimum 
clearances  for  the  different  vertical  velocities.  As  explained  by  Rueb  &  Hassoun  (1990), 
the  x-axes  for  the  two  aircraft  reflect  much  of  the  differences  between  the  windows.  The 
F-l  1 1  x-axis  is  in  hundreds  of  feet  per  second  (tens  of  thousands  of  feet  per  minute); 
whereas,  the  KC-135  x-axis  is  in  thousands  of  feet  per  minute. 

Future  efforts  should  be  directed  at  identifying  pilot  windows  of  acceptability  to  be 
used  in  the  development  of  a  GCAS  system.  These  windows  would  provide  both  the 
developer  and  the  SPO  with  an  accurate  method  for  evaluating  the  false  alarm  and 
prediction  accuracy  rates  for  a  given  GCAS  system.  Additional  GCAS  evaluations  should 
also  include  man-in-the  loop  simulations  interfaced  with  actual  terrain  databases.  As 
shown  by  this  study,  this  provides  a  realistic  and  informative  method  for  evaluating  the 
GCAS  algorithm. 
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FB- 1 1 1  PILOT  WINDOW  OF  ACCEPTABILITY 


1 1  A 


L  Pilot  window  of  acceptability  for  FB-111  aircraft. 


KC-135  PILOT  WINDOW  OF  ACCEPTABILITY 
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Figure  84,  Pilot  window  of  acceptability  for  KC-135  aircraft. 
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I.  INTRODUCTION 


The  following  report  describes  cargo/transport/tanker  Controlled  Flight  Into  Terrain 
(CFIT)  mishaps  covering  the  period  of  1970-Present.  During  this  period,  thirty-one  CFIT 
mishaps  occurred  that  involved  cargo/transport/tanker  type  aircraft.  The  mishap  aircraft 
included;  12  C-130's,  7  C-141's,  5  C/KC-135's,  C-7,  C-9,  C-12,  C-54,  C-123,  C-124, 
and  C-140.  Each  mishap  was  classified  into  various  categories  (e.g.,  low  level, 
mountainous  terrain,  night,  etc.)  to  identify  any  trends.  A  brief  statistical  synopsis  of  these 
mishaps  follows. 


2.  GCAS  SYSTEM  IMPACT 

Figure  A-l  presents  a  breakdown  of  the  CFIT  mishaps  into  three  categories  based 
on  whether  an  operational  Ground  Collision  Avoidance  System  (GCAS)  could  have 
prevented  such  an  occurrence.  The  three  categories  are  No  Help,  May  Help,  and  Help. 
Two  CSEF  engineers  determined  these  classifications  based  on  the  1970-Present  CFIT 
accident  mishap  reports.  A  No  Help  classification  resulted  when  it  was  determined  that  an 
operable  GCAS  system  would  not  have  provided  a  warning  signal  (e.g.,  aircraft  striking  a 
tall  tree,  aircraft  hitting  the  side  of  a  sheer  cliff/mesa).  A  May  Help  classification  occurred 
when  no  exact  determination  could  be  made  from  the  mishap  report  findings.  An  aircraft 
hitting  the  ground  after  acknowledging  level  at  a  given  altitude  or  crossing  a  given  tacan  fix 
is  an  example  of  a  May  Help  rating.  A  Help  classification  resulted  when  it  appeared 
obvious  that  the  mishap  could  have  been  prevented,  given  a  usable  GCAS  onboard  (e.g., 
aircraft  crashes  into  the  top  of  a  mountain  peak  due  to  aircrew  inattention).  It  was  found 
that  13  accidents  (42%)  could  not  have  been  prevented  by  GCAS,  whereas,  18  (8  May 
Help  26%,  10  Help  32%)  mishaps,  costing  numerous  lives  and  millions  of  dollars,  may 
have  been  prevented  by  an  operable  GCAS  system.  Thus,  there  is  ample  justification  for 
installation  of  a  Ground  Collision  Avoidance  System  in  the  C/KC-135  fleet. 

3.  CLASS  A  MISHAP  ANALYSIS  AND  RESULTS 

Figure  A-2  shows  the  breakdown  of  the  mishaps  according  to  the  phase  of  flight 
and  the  effectiveness  of  the  GCAS  system.  The  rectangles  with  stripes  canted  right 
represent  GCAS  preventable  mishaps.  The  left-canted  striped  rectangles  represent  those 
mishaps  where  GCAS  would  not  have  been  effective.  The  approach  and  landing  phase 
accounts  for  50%  of  the  mishaps;  the  low  level  phase,  33%.  The  takeoff  and  cruise  phases 
together  account  for  only  17%  of  the  mishaps,  half  that  of  the  low  level  phase. 
Accordingly,  mission  scenarios  shall  focus  on  the  low  level  and  the  approach/landing 
phases  of  flight. 

Figure  A-3  categorizes  the  mishaps  according  to  type  of  terrain  where  the  aircraft 
mishap  occurred.  Of  the  preventable  mishaps,  mountainous  terrain  mishaps  equalled  flat 
and  rolling  mishaps.  This  finding  may  be  biased  toward  the  flat  and  rolling  terrain.  In 
determining  type  of  terrain,  it  was  assumed  that  all  accidents  not  specifically  mentioning  a 
peak  or  mountainous  type  terrain  occurred  in  flat  and  rolling  terrain.  This  assumption  fails 
to  account  for  the  possible  oversight  of  the  mishap  investigator  not  reporting  a 
mountainous  condition  when  it  might  have  existed.  Accordingly,  mission  scenarios  will 
include  mountainous  and,  flat  and  rolling  terrain. 

Day  and  nighttime  conditions  are  represented  by  Figure  A-4.  Daytime  accounted 
for  twice  as  many  mishaps  than  nighttime  conditions.  This  suggests  the  need  to  investigate 
daytime  condition.  However,  further  investigation  reveals  that  die  majority  of  flights 
(approximately  75%)  occur  during  daytime  conditions.  Therefore,  daytime  conditions 
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should  be  triple  that  of  the  nighttime  conditions,  but  are  only  twice  the  number  of  nighttime 
mishaps.  Accordingly,  mission  scenarios  should  focus  on  both  daytime  and  nighttime 
conditions. 

Daytime  and  night  time  conditions  are  further  divided  into  categories  based  on 
whether  adverse  weather  conditions  existed  (Figure  A-5).  This  breakdown  reveals  that 
GCAS  preventable  mishaps  were  equally  split  between  clear  and  adverse  weather 
conditions  for  both  the  daytime  and  nighttime  categories.  In  a  similar  fashion  to  day/night 
flights,  this  finding  is  slightly  deceiving.  A  larger  percentage  of  flights  is  conducted  during 
clear  sky  conditions  than  during  adverse  weather  conditions,  because  of  ICAO/FAA 
flight  safety  restrictions. 

Figure  A-6,  secondary  causes,  identifies  those  mishaps  that  involved  other  factors 
that  may  have  contributed  to  the  mishap.  Two  incidents  involved  malfunctioning  aircraft, 
while  six  involved  air  traffic  controller  instructions  (e.g.,  aircraft  29,  cleared  descent  to 
4000  feet).  Of  the  six  controller  involved  incidents,  three  (50%)  involved  instructions  to 
maintain  altitude  or  descend  to  altitudes  that  were  below  minimum  safe  clearance  altitudes. 
A  fourth  mishap  resulted  from  an  aircrew  incorrectly  acknowledging  a  controller's 
clearance  altitude.  No  malfunction  related  mishaps  would  have  been  avoidable. 

4.  CONCLUSION 

Based  on  Figures  A-l  through  A-6,  an  operable  Ground  Collision  Avoidance 
System  would  possibly  have  prevented  58%  of  die  Controlled  Flight  Into  Terrain  mishaps. 
It  also  suggests  that  three  GCAS  modes  might  be  beneficial:  (1)  Takeoff,  (2)  Low  Level, 
and  (3)  Approach/Landing.  Additionally,  these  figures  indicate  that  mission  scenarios 
should  focus  on  the  low  level  and  approach/landing  phases  of  flight,  day  and  nighttime 
conditions,  and  both  flat  anu  rolling  and  mountainous  type  terrain.  Thus,  our  evaluation 
shall  focus  on  this  type  of  mission  scenario. 


120 


35/CARGO/TANKER 


KC  —  1  35/CARGO /TANKER  TYPE 


KC- 1  35/CARGO/TANKER  TYPE  CFIT  ACCIDENTS 
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METEOROLOGICAL  CONDITIONS 
~Z_ / 1  GCAS  PREVENTABLE  [\  \J  GCAS 
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GROUND  COLLISION  AVOIDANCE  SYSTEM 
QUESTIONNAIRE  FOR 
CARGO/TANKER  TYPE  AIRCRAFT 


TANKER  MODERNIZATION  DIVISION 
DIRECTORATE  OF  BOMBER/TANKER 
AERONAUTICAL  SYSTEMS  DIVISION 
AIR  FORCE  SYSTEMS  COMMAND 

WRIGHT-PATTERSON  AIR  FORCE  BASE, OHIO  45433-6503 
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PERSONAL  DATA 


Name  (Optional): _ 

Rank: _ 

Aeronautical  Rating: _ 

Organization: _ 

Office  Symbol: _ 

Duty  Station: _ 

Crew  Position: _ 

Total  Flying  Hours: _ 

Total  KC-135  Flying  Hours: _ 

Total  Hours  Current  Crew  Position: _ 

Age: _  Sex: _ 

Describe  any  prior  experience  with  Ground  Collision  Avoidance  Systems 
(GCAS)  or  Ground  Proximity  Warning  System(GPWS): _ 
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GENERAL  INFORMATION 


1.  Would  you  consider  a  Ground  Collision  Avoidance  System  (GCAS)  to 
be  beneficial  for  warning  an  aircrew  of  a  possible  ground  collision? 

(a)  Yes 

(b)  No 

(c)  Maybe  (Please  explain  in  comments) 

Comment(s): _ 


2.  Which  of  the  following  GCAS  warning  modes  would  you  consider  as 
most  effective  in  attracting  an  aircrew  member's  attention  in  a  cockpit 
environment? 


(a)  Light 

(b) Tone 

(c)  Voice 

(d)  A  combination  of  light  and  tone 

(e)  A  combination  of  light  and  voice 

(f)  A  combination  of  tone  and  voice 

(g)  Other  (Please  specify) 

Comment(s): _ 


3.  Would  you  consider  different  GCAS  modes  (e.g.,  low  level,  takeoff, 
wind  shear,  etc.)  that  adjust  for  the  different  phases  of  flight  as  beneficial? 

(a)  Yes 

(b)  No 

(c)  Maybe  (Please  explain  in  comments) 

Comment(s): _ 
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4.  Which  of  the  following  GCAS  modes  would  you  consider  as 
beneficial?  Select  any  or  all  of  the  answers  that  you  think  apply.  If  you 
feel  mode(s)  other  than  those  listed  is/are  necessary,  choose  item  (f)  and 
specify  in  the  comments  section. 


‘J 

(a)  AppjHJ. .  tending 

(b)  LowLc  ‘A 

(c)  Rapid  Descent 

(d)  Takeoff 

(e)  Wind  Sheer 

(f)  Other  (Please  specify) 


Comment(s):. 


5.  Given  the  existence  of  different  GCAS  modes,  please  prioritize  the 
following  modes  from  most  beneficial  to  least  beneficial  in  the  comments 
section?  If  you  feel  another  mode  is  necessary,  choose  item  (other)  and 
specify  what  mode  and  why.  (e.g.,  Approach/Landing  1,  Low  Level  2, 
Rapid  Descent  3,  Takeoff  4,  Wind  Sheer  5,  Other 
(Over  Water-because  .  .  .)  6). 

Approach/Landing 

Low  Level 

Rapid  Descent 

Takeoff 

Wind  Sheer 

Other  (Please  specify) 

Comment(s): _ 


6.  Should  the  pilot  be  able  to  turn  off  the  GCAS? 

(a)  Yes 

(b)  No 

(c)  Maybe 

Comment(s): _ 
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7.  What  should  the  pitch  limits  of  the  GCAS  be? 

Minimum  (Lower)  limit _ Maximum  (Upper)  limit 

Comment(s): _ 


8.  What  should  the  roll  limits  of  the  GCAS  be? 

Minimum  (Lower)  limit _ Maximum  (Upper)  limit 

Comment(s): _ 


9.  Should  the  GCAS  extrapolate  beyond  the  range  of  the  radar  altimeter? 

(a)  Yes 

(b)  No 

Comment(s): _ 


10.  What  should  the  maximum  altitude  coverage  of  the  GCAS  be? 

(a)  Maximum  coverage  of  the  radar  altitude. 

(b)  5000  feet 

(c)  10.000  feet 

(d)  Maximum  altitude  (ceiling)  of  the  aircraft. 

Comment(s): _ 
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II.  What  should  the  GCAS  minimum  descent  altitude  (altitude  where  the 
GCAS  warning  is  inhibited)  be? 

(a)  0  feet 

(b)  50  feet 

(c)  100  feet 

(d)  200  feet 

(e)  Other  (please  specify) 

Comment(s): _ 


VISUAL  MODE 


1.  What  type  of  light  would  you  consider  to  be  most  beneficial  in  alerting 
the  aircrew  of  a  possible  ground  collision? 


(a)  Flashing 

(b)  Steady 

(c)  Other  (Please  specify) 

Comment(s): _ 


2.  What  nomenclature  should  be  printed  on  the  warning  light? 

(a)  Altitude 

(b)  Climb 

(c)  GCAS 

(d)  Pull  up 

(e)  Recover 

(f)  Other  (Please  specify) 

Comment(s): _ 
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3.  How  long  should  the  warning  light  be  present? 


(a)  1  second  (s) 

(b)  2  s 

(c)  3  s 

(d)  4  s 

(e)  5  s 

(f)  As  long  as  the  warning  condition  exists. 

(g)  Other  (Please  specify) 

Comment(s): _ 


4.  If  a  flashing  light  is  chosen,  what  time  interval  between  warnings 
would  you  consider  optimal  (e.g.,  light,  1  second  interval,  light)? 

(a)  .1  s 

(b)  .5  s 

(c)  1.0  s 

(d)  Other  (Please  specify) 

Comment(s): _ 


AUDITORY  MODE 

TONE  WARNING 

1.  What  type  of  tone  would  you  consider  to  be  most  beneficial  for  alerting 
the  aircrew  of  a  possible  ground  collision? 

(a)  Intermittent 

(b)  Steady 

(c)  Wavering 

(d)  Alternating  between  steady  and  wavering  tone 

(e)  Other  (Please  specify) 

Comment(s): _ 
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2.  How  long  should  each  warning  signal  be  present? 

(a)  1  second  (s) 

(b)  2  s 

(c)  3  s 

(d)  As  long  as  the  warning  condition  exists. 

(e)  Other  (Please  specify) 

Comment(s): _ 


3.  For  an  intermittent  tone,  what  time  interval  between  warnings  would 
you  consider  optimal  (e.g.,  tone,  1  second  interval  with  no  tone,  tone)? 

(a)  .1  s 

(b)  .5  s 

(c)  1.0  s 

(d)  Other  (Please  specify) 

Comment(s): _ 


VOICE  WARNING 

1.  What  type  of  voice  should  it  be? 

(a)  Computerized  female 

(b)  Computerized  male 

(c)  Human  female 

(d)  Human  male 

(e)  Other  (Please  specify) 


Comment(s):. 


2.  What  voice  message  would  you  consider  to  be  most  effective  in  alerting 
the  aircrew  of  a  possible  ground  collision? 

(a)  Altitude 

(b)  Climb 

(c)  Pull  up 

(d)  Recover 

(e)  A  combination  of  two  or  more  messages  listed  above  (Please  specify) 

(f)  Other  (Please  specify) 

Comment(s): _ 


3.  How  many  times  should  the  voice  warning  be  presented? 

(a)  One  (e.g.,  altitude) 

(b)  Two  (e.g.,  climb,  climb) 

(c)  Three  (e.g.,  pull  up,  pull  up,  pull  up) 

(d)  Until  the  conditions  to  affect  recovery  are  completed. 

(e)  Other  (Please  specify) 

Comment(s): _ 


4.  What  time  interval  between  warnings  would  you  consider  optimal 
(ex:  "Recover",  1  second  interval,  "Recover")? 

(a)  .1  s 

(b)  .5  s 

(c)  1.0  s 

(d)  Other  (Please  specify) 

Comment(s): _ _ 
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MINIMUM  ACCEPTABLE  CLEARANCE  ALTITUDE 


The  Minimum  Acceptable  Clearance  Altitude  (MACA)  graph  is  on  the  following 
page.  This  graph  represents  the  relationship  between  what  a  pilot  feels  is  the  minimum 
acceptable  clearance  altitude  (Above  Ground  Level- AGL)  given  the  aircraft's  downward 
vertical  velocity  (Vz).  MACA  is  the  lowest  altitude  (AGL)  that  an  aircraft  reaches 
during  recovery  from  the  initial  GCAS  warning  for  which  a  pilot  feels  minimum  stress. 

For  example,  if  the  alarm  sounds  with  a  Vz  of  600  feet  per  minute  (fjpm),  the  minimum 
acceptable  clearance  altitude  for  a  given  pilot  might  be  150  feet  (AGL),  whereas,  at  a  Vz  of 
6000  fpm,  it  might  be  1000  feet.  (This  is  only  an  example,  the  numbers  do  not  reflect  any 
sort  of  experimentally  collected  data.)  We  would  like  you  to  draw  a  line  that  best 
approximates  your  minimum  acceptable  clearance  altitudes  for  the  given  downward  vertical 
velocity  onto  die  graph.  Please  give  this  careful  consideration  and  take  enough  time  in 
drawing  the  line.  Please  draw  direcdy  on  the  graph  and  answer  the  following  question. 

1.  Do  you  fully  understand  what  the  minimum  acceptable  clearance  altitude 
is  and  how  to  draw  it  onto  the  graph? 

(a)  I  fully  understand  what  MACA  is  and  how  to  plot  it. 

(b)  I  fully  understand  what  MACA  is  but  am  uncertain  how  to  plot  it 

(c)  I  do  not  understand  what  MACA  is  and  am  uncertain  how  to  plot  it. 

(d)  Other  (Please  specify) 

Comment(s): _ 
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MINIMUM  ACCEPTABLE  CLEARANCE  ALTITUDE 

AS  A  FUNCTION  OF  VERTICAL  VELOCITT  (Vzl 


(spuvsnoyjj 

HOY)  apn>/w  bouvjv&iq  uinwiuw 


138 


(Thousands) 

Downward  Vertical  Velocity  (Vz) 


APPENDIX  C 

PILOT  GCAS  QUESTIONNAIRE 
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1.  In  your  opinion,  the  overall  ground  clearance  provided  by  this  Ground 
Collision  Avoidance  System  (GCAS)  algorithm  was: 

a.  Too  high. 

b.  Slightly  high. 

c.  About  right. 

d.  Slightly  low. 

e.  Too  low. 

Comments: _ 


2.  This  GCAS  algorithm  applied  its  warnings: 

a.  Very  consistently. 

b.  Consistently. 

c.  Inconsistently. 

d.  Very  inconsistently. 

Comments: _ 


3.  How  should  the  warning  be  mechanized?  From  the  time  of  warning  to: 

a.  One  iteration  of  the  warning. 

b.  Two  iterations  of  the  warning. 

c.  Until  the  aircraft's  flight  path  is  level  with  terrain  (radar  vertical  velocity  =  0). 

d.  Until  some  positive  radar  vertical  vertical  velocity  is  attained. 

e.  For  as  long  as  the  warning  conditions  exist. 

Comments: _ _ 
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4.  What  type  of  warning(s)  do  you  think  would  be  best  for  this  GCAS? 


a.  Light 

b.  Tone 

c.  Voice 

d.  Combination  of  light  and  tone. 

e.  Combination  of  light  and  voice. 

f.  Combination  of  tone  and  voice. 

Comments: _ 


5.  In  your  opinion,  the  job  of  a  ground  collision  avoidance  system  is  done 
when  : 

a.  The  pilot  is  made  aware  of  the  situation. 

b.  The  pilot  initiates  corrective  action. 

c.  A  flight  path  level  with  the  terrain  is  attained. 

d.  A  positive  radar  altitude  rate  is  achieved. 

Comments: _ 


6.  How  confident  are  you  in  this  Gri  id  Collision  Avoidance  System 
algorithm? 

a.  Very  confident 

b.  Confident. 

c.  Not  confident. 

d.  No  confidence. 

Comments:. _ 
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7.  In  your  opinion,  this  algorithm  you've  just  flown  is  suitable  for: 

a.  Training  missions  only. 

b.  Peacetime  operational  missions. 

c.  Combat  operations. 

d.  All  of  the  above. 

e.  None  of  the  above. 

Comments: _ 


8.  In  your  opinion,  would  the  KC-135  aircraft  fleet  benefit  from  the  use  of 
the  Cubic  GCAS  Algorithm?  (Please  explain  why  or  why  not.) 

a.  Yes 

b.  No 

c.  Maybe. 

Comments: _ 


9.  Please  rate  the  overall  performance  of  the  KC-135  simulator  operation. 

a.  Excellent. 

b.  Good. 

c.  Fair. 

d.  Poor. 

Comments: _ 
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